-- Logs begin at Wed 2022-08-10 12:32:55 UTC, end at Wed 2022-08-10 13:46:47 UTC. -- Aug 10 13:05:53.644853 dv-cinder-836027-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 13:05:53.655729 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 13:05:53 2022] *** Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: nodename: dv-cinder-836027-5-iscsi Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: machine: x86_64 Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: clock source: unix Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: pcre jit disabled Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: detected number of CPU cores: 4 Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: current working directory: / Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: detected binary path: /usr/bin/uwsgi-core Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: your processes number limit is 47680 Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: your memory page size is 4096 bytes Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: detected max file descriptor number: 2048 Aug 10 13:05:53.658467 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: lock engine: pthread robust mutexes Aug 10 13:05:53.659189 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: thunder lock: enabled Aug 10 13:05:53.659189 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 13:05:53.659189 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 13:05:53.698521 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: Python main interpreter initialized at 0x55f55cf439c0 Aug 10 13:05:53.698521 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: python threads support enabled Aug 10 13:05:53.698521 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: your server socket listen backlog is limited to 100 connections Aug 10 13:05:53.698521 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: your mercy for graceful operations on workers is 90 seconds Aug 10 13:05:53.698928 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 13:05:53.698928 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: *** Operational MODE: preforking *** Aug 10 13:05:53.699270 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: *** uWSGI is running in multiple interpreter mode *** Aug 10 13:05:53.699270 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: spawned uWSGI master process (pid: 80212) Aug 10 13:05:53.699579 dv-cinder-836027-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 13:05:53.702301 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: spawned uWSGI worker 1 (pid: 80227, cores: 1) Aug 10 13:05:53.702301 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: spawned uWSGI worker 2 (pid: 80228, cores: 1) Aug 10 13:05:53.702301 dv-cinder-836027-5-iscsi devstack@keystone.service[80212]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 13:05:59.321136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:05:59.321136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: from cryptography.utils import int_from_bytes Aug 10 13:05:59.321927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:05:59.321927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: from cryptography.utils import int_from_bytes Aug 10 13:05:59.322347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:05:59.322347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: from cryptography.utils import int_from_bytes Aug 10 13:05:59.322455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:05:59.322455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: from cryptography.utils import int_from_bytes Aug 10 13:05:59.371325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 13:05:59.371509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 13:05:59.371688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] command line args: [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 13:05:59.371955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 13:05:59.372132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ================================================================================ {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 13:05:59.372216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 13:05:59.372358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 13:05:59.372445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] admin_token = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.372553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] command line args: [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 13:05:59.372631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] config_dir = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.372784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.372862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 13:05:59.372964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] config_source = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ================================================================================ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 13:05:59.373159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] debug = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] admin_token = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config_dir = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.373962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config_source = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] debug = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.374738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_config_append = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_dir = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_file = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.375975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_config_append = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_dir = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.376952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.383564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.383661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.383791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.383869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.383946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.384953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] publish_errors = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] publish_errors = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.385978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] standard_threads = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] standard_threads = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] use_journal = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] use_json = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.386980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] use_stderr = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] use_syslog = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_journal = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_json = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.387624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_stderr = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.387763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_syslog = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.387910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:05:59.387985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.388852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.389963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.390976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.391961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.392966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393349 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.393939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.394996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.395994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.396933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.397925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.398946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.399970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.400781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.401996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.402916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.403878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.404950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.405981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.406725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.406817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.406976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.407966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.external = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.external = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.408955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.password = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] auth.token = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.password = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.409913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.token = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.410866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.411978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.412943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.413945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.414949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.415929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.416967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.417958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.418917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.419859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.420958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.421838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.422960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.423957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.424983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.425956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.426901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.427985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.428922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.429976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.430944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.431899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.432841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.433953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.434906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.435911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.436857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.437903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.438999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.439936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.440982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] role.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.441960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] role.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.442901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.443889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.444844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.445906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.446896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.447999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.448943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449295 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.449938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.450964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.451913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.452873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.453664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.453886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.453956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.454941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.connection = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.455911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.456917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.457999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.458948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.459806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80227) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 13:05:59.459886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.460097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.460344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.460582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:05:59.460754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 13:05:59.488968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.489229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.489453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.490455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.490689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.490915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.499103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.500928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.528026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.528252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.529449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.529684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.583695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.584498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.634220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.634438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.634531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.634673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.700432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.700645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.700772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.700859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.700936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.701737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.724810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.725931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.726013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.726141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.726222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.726351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.737446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.737619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.750590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.750813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.750970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.751874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80227) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.752005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.752216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:05:59.769055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.769265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:05:59.769513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.769595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.769726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:05:59.769807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:05:59.770053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.770134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.770257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:05:59.770337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.770537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.770625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.770802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.770881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.771061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.771318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.782969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.783245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.783517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.783677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.783797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.783960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.784234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.784517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.790682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.791519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.793728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.794013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.794291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.794570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.794650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.794849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.794930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795919 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.795999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.796188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.796267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.796454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.796535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.797337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.797826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.798164 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.820377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.821253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.821891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.822180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.822747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.823045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.826746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.827032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.827312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.827652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.827962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.828248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.834189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.834429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.834828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.835120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.838525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.838820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.839101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.839257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.839424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.839562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.839661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.839853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.839990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.840140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.840223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.840427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.840511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.840712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.841011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.841290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.856928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.857234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.857835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.858117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.861886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.862655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.865014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.865326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.865821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.866118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.869690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.870523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.872851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.873730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.876064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.876364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.876660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.876957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.877087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.877268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.877362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.877546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.877631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.877834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.878116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.878410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.893569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.893866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.894136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.894401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.894484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.894779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.895051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.895318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.901487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.902464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.903049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.903326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.904032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.904313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.906652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.907032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.907324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.907616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.907699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.907893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.907977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.908210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.908503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.908784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.919213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.919498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.919775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.920052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.920191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.920330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.920478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.920607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.920759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.920881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.921064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.921350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.921630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.922461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.938319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.939742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.941289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.942749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.944214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.945690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.945812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.947268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.947364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.947591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.947882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.948762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.949086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.949382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.954354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.955829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.955924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.957474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.958870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.959160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.959443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.959717 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.960413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.960706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.961014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.961291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.967726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.968110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.969082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.969902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.970193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.970472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.976031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80227) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.976319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.976598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.976877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.977174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.977348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:05:59.977433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.977635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.977731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.977920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.978875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80227) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.979963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.980242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.980507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.980784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:05:59.981070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:06:00.013224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f55cf439c0 pid: 80227 (default app) Aug 10 13:06:00.014786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f55cf439c0 pid: 80228 (default app) Aug 10 13:06:00.016624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52067334-c781-4915-8eae-36e35dbacf18 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:00.016920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52067334-c781-4915-8eae-36e35dbacf18 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:00.017219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52067334-c781-4915-8eae-36e35dbacf18 None None] PATH_INFO: `/v3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:00.021448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Wed Aug 10 13:06:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:08.672101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:08.672565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:08.672655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:08.677047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:08.682199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:08.682627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:08.682746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:08.754205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: INFO dbcounter [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] Registered counter for database keystone Aug 10 13:06:08.773992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG oslo_db.sqlalchemy.engines [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80227) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 13:06:08.780523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writer thread running {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 13:06:08.959682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG passlib.handlers.bcrypt [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80227) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 13:06:08.960029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG passlib.handlers.bcrypt [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80227) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 13:06:09.015025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:09.016648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:09.146612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 13:06:09.296256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:09.370482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: INFO dbcounter [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] Registered counter for database keystone Aug 10 13:06:09.391989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG oslo_db.sqlalchemy.engines [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80228) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 13:06:09.398100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writer thread running {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 13:06:09.515859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d17d9-68de-4335-b00d-3e9b8a74d476 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:09.520078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:09.520696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:09.520976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:09.521269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:09.529477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:09.582129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 13:06:09.582412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7d83a-3d11-46d9-a47c-43a0aabafc10 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:09.591793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Aug 10 13:06:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:06:09.598465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:09.612659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2990eb8-f72f-4c57-bb8c-b4ab2fa99bcc None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:09.617119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:09.618204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:09.618490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:09.618775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] PATH_INFO: `/v3/projects/55c6cff78b754392afc788fabbf074a6` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:09.620187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] RBAC: Authorizing `identity:get_project(project_id=55c6cff78b754392afc788fabbf074a6)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:09.671300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING keystone.common.rbac_enforcer.enforcer [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 13:06:09.671797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a464-18c8-4dce-baa4-07093805a086 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:09.676473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Wed Aug 10 13:06:09 2022] GET /identity/v3/projects/55c6cff78b754392afc788fabbf074a6 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:11.667557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fb3f-50fd-4947-84a3-0feb5d842222 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:11.667997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fb3f-50fd-4947-84a3-0feb5d842222 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:11.668094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fb3f-50fd-4947-84a3-0feb5d842222 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:11.672231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:11.677546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:11.677971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:11.678095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:11.699099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:11.700608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:11.709546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 13:06:11.732598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fb3f-50fd-4947-84a3-0feb5d842222 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:11.744597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fb3f-50fd-4947-84a3-0feb5d842222 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:11.748783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d513d81-905c-4203-887a-b5144eb93e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:11.749408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d513d81-905c-4203-887a-b5144eb93e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:11.750196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d513d81-905c-4203-887a-b5144eb93e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:11.750468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d513d81-905c-4203-887a-b5144eb93e6e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:11.758025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d513d81-905c-4203-887a-b5144eb93e6e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:11.759804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d513d81-905c-4203-887a-b5144eb93e6e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:11.769098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Aug 10 13:06:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:06:11.774503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:11.784297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f219884-d62f-4052-a51c-9fcc0cc04b2d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:11.788577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2678c23-a43c-4c65-b2d9-f01b9e43f0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:11.789225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2678c23-a43c-4c65-b2d9-f01b9e43f0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:11.789506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2678c23-a43c-4c65-b2d9-f01b9e43f0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:11.789777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2678c23-a43c-4c65-b2d9-f01b9e43f0c4 admin admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:11.791085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2678c23-a43c-4c65-b2d9-f01b9e43f0c4 admin admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:11.792976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2678c23-a43c-4c65-b2d9-f01b9e43f0c4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:11.794627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Wed Aug 10 13:06:11 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:13.860454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048ed4-eeea-461a-933c-1f941dfeac5c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:13.861557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048ed4-eeea-461a-933c-1f941dfeac5c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:13.861646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048ed4-eeea-461a-933c-1f941dfeac5c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:13.862989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:13.867918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:13.868312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:13.868467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:13.883237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ee66c-846a-44d0-b6e3-e1e2cd9e94a5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:13.883519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ee66c-846a-44d0-b6e3-e1e2cd9e94a5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:13.883799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ee66c-846a-44d0-b6e3-e1e2cd9e94a5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:13.885113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:13.886302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:13.887506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:13.889889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:13.890173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:13.890453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:13.934172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG passlib.handlers.bcrypt [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80228) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 13:06:13.934509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG passlib.handlers.bcrypt [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80228) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 13:06:13.968675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:13.980418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:13.982028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:13.991730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.015413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.082961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:14.094803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f619d-9ab9-47ae-b6b7-d1893ad86280 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.104179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.130441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.131087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.131372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.131649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.143767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:14.152388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:14.153352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:14.154189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:14.157137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:14.159564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.210811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d5c5c-3a5d-4e7c-8813-84c82a738aa5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.213650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.241057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.249208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.250070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.250359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.250642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.259081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee2f00-68e2-4f3a-a905-a362b7b485e8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.293681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.294302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.294955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.294955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.295495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:14.295798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.297220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.309478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:14.315211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.322378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.323905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.416526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d460621-ce88-4100-a766-8fdba8778b36 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.421055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:06:14.427058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.430802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.535595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69f533-2175-4e05-941c-02df19a70528 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.560038 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.560679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.560972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.561281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.562425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.564595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.566388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:14.571731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.575246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.575246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.575246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.575246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.588368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.594586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.598179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:06:14.614897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:14.615772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:06:14.616561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:06:14.616561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:06:14.616561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:14.616561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:14.622064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.679279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b749ac-8e4d-4d3b-b0f3-9e1451a35ee2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.680403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.713352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.713991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.714276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.714557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.715723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.717206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.718940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:14.725016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:14.727239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bc26f-0c4e-49c7-afd0-b4234a025dba None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.761231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.761865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.762147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.762416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.763053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:14.763352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.764754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375620d-e94f-41ad-a79c-c62e874b21c0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.783053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:14.807754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:14.827790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0e64-852f-48a1-affc-8a265c3a88fd None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:14.866644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:14.867267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:14.867550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:14.867826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:14.868575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.domain.id=default, user.id=40886c602f2b4c218564e5847aed724b) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:14.869144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:14.870366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382660f-92a1-4106-965d-92ad0e0c321f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:14.878275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Aug 10 13:06:14 2022] GET /identity/v3/role_assignments?user.id=40886c602f2b4c218564e5847aed724b&scope.domain.id=default&role.id=dd76871a868143b482c4982027fdc1f5 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:16.961924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1702b6b-ca0e-45f3-8d01-f45db67ccb2a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:16.962402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1702b6b-ca0e-45f3-8d01-f45db67ccb2a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:16.962402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1702b6b-ca0e-45f3-8d01-f45db67ccb2a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:16.963728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:16.969179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:16.969575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:16.969741 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:16.998049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:16.999289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:17.087591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 12/23] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:17.123319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1702b6b-ca0e-45f3-8d01-f45db67ccb2a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:17.254044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1702b6b-ca0e-45f3-8d01-f45db67ccb2a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:17.277523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:17.300668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ae2fd-28b2-4a9a-8502-8b8b384726be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:17.307202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ae2fd-28b2-4a9a-8502-8b8b384726be None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.307202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ae2fd-28b2-4a9a-8502-8b8b384726be None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.307202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ae2fd-28b2-4a9a-8502-8b8b384726be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.318240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ae2fd-28b2-4a9a-8502-8b8b384726be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4f5ae2fd-28b2-4a9a-8502-8b8b384726be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:17.324718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:17.325571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:17.326327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:17.328231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 12/24] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:17.334748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:17.368307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb861-b433-4067-810f-bb2f7099b22f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.368755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb861-b433-4067-810f-bb2f7099b22f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.369179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb861-b433-4067-810f-bb2f7099b22f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.370624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 13/25] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:17.375498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.375893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.376328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.395027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:17.396361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:17.505446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:17.516360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 14/26] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:17.542967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:17.560455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58180c91-8967-481b-8577-69b6cfad7521 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:17.646375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:17.647018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.653024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.653024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.653024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:17.653024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:17.653024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c501fe-ff90-47ff-bb9d-1ba7b033ecc1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:17.679523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 13/27] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:17.680849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba2e81-c850-46b1-b5fd-c6ac7b58af44 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.681187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba2e81-c850-46b1-b5fd-c6ac7b58af44 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.681468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba2e81-c850-46b1-b5fd-c6ac7b58af44 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.683175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 14/28] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:17.687705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba2e81-c850-46b1-b5fd-c6ac7b58af44 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:17.816494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7893ee-69d1-4f25-82b0-ad963af82a26 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:17.827384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:17.833443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba2e81-c850-46b1-b5fd-c6ac7b58af44 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:17.866883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1da35-cbf1-4284-be43-f4416facf0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:17.868103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1da35-cbf1-4284-be43-f4416facf0f3 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.869012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1da35-cbf1-4284-be43-f4416facf0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.869750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1da35-cbf1-4284-be43-f4416facf0f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.870858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1da35-cbf1-4284-be43-f4416facf0f3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:17.873180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1da35-cbf1-4284-be43-f4416facf0f3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:17.877028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29859-85b2-47fc-9648-b98b238cf86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:17.877893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29859-85b2-47fc-9648-b98b238cf86c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.878330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29859-85b2-47fc-9648-b98b238cf86c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.878810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29859-85b2-47fc-9648-b98b238cf86c None admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.880191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad29859-85b2-47fc-9648-b98b238cf86c None admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:17.882222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad29859-85b2-47fc-9648-b98b238cf86c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:17.883909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 15/29] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:17 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:17.885231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e8e4a-f4b5-443b-a8cd-88493fea2f8e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.885651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e8e4a-f4b5-443b-a8cd-88493fea2f8e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.886050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e8e4a-f4b5-443b-a8cd-88493fea2f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.893985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:17.906020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-dc7e8e4a-f4b5-443b-a8cd-88493fea2f8e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:17.907224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dc7e8e4a-f4b5-443b-a8cd-88493fea2f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:17.974709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 16/30] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:17.976042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4886ee-cb1e-4530-ac8e-40276cae7240 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:17.976333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4886ee-cb1e-4530-ac8e-40276cae7240 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:17.976612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4886ee-cb1e-4530-ac8e-40276cae7240 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:17.978086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 17/31] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:17.978798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4886ee-cb1e-4530-ac8e-40276cae7240 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:18.102244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4886ee-cb1e-4530-ac8e-40276cae7240 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:18.102440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 15/32] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 10 13:06:17 2022] POST /identity/v3/roles => generated 219 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:06:18.103873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.104161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.104440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.125647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:18.127404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:18.165326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:18.166375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.166956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.167576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.169836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:18.171574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:18.174495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 18/33] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:18.180480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:18.184357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:18.281480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 16/34] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:18.282356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:18.309904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f438d-b4ba-4622-bb0a-d1bd299bec9c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:18.365495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:18.366108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.366386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.366671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.382268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:18.395829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:18.402789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:06:18.404638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:18.405535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:06:18.406323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:06:18.406323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:06:18.406323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:18.406323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 19/35] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:18.407042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:18.414095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233427e-a805-4f1f-88a0-75f7a0492399 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:18.463051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:18.463962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.464414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.464829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] PATH_INFO: `/v3/domains/default/users/40886c602f2b4c218564e5847aed724b/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.468126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=40886c602f2b4c218564e5847aed724b, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:18.471073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:18.489324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:18.543133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f825-37f1-46d7-a397-9bd9dbb69f2b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:18.577509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 17/36] 10.222.0.40 () {58 vars in 1361 bytes} [Wed Aug 10 13:06:18 2022] PUT /identity/v3/domains/default/users/40886c602f2b4c218564e5847aed724b/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:18.578463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:18.584109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4a60f-5843-473c-b877-c32bde14e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:18.585192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4a60f-5843-473c-b877-c32bde14e48d None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.585697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4a60f-5843-473c-b877-c32bde14e48d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.586168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4a60f-5843-473c-b877-c32bde14e48d None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.587072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4a60f-5843-473c-b877-c32bde14e48d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:18.589596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4a60f-5843-473c-b877-c32bde14e48d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:18.708388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821d8cb-7d17-48ab-b255-0dd24e2acb02 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:18.750193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:18.750823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.751882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.752372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.753233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:18.753545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:18.755055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd7737-ebb1-4e00-a790-0b2180af4a63 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:18.768532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 18/37] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:18.769970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97dd4-4bb3-4cea-8911-b839c39b0119 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.770262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97dd4-4bb3-4cea-8911-b839c39b0119 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.770540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97dd4-4bb3-4cea-8911-b839c39b0119 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.771861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 19/38] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:18.775007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97dd4-4bb3-4cea-8911-b839c39b0119 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:18.795824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:18.850197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 20/39] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 10 13:06:18 2022] POST /identity/v3/roles => generated 225 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:18.852009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de8ed-7f2d-4097-a57e-96d06c93d50f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.852399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de8ed-7f2d-4097-a57e-96d06c93d50f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.852778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de8ed-7f2d-4097-a57e-96d06c93d50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.886184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d40de8ed-7f2d-4097-a57e-96d06c93d50f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:18.887772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d40de8ed-7f2d-4097-a57e-96d06c93d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:18.905425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97dd4-4bb3-4cea-8911-b839c39b0119 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:18.953645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:18.955803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.956300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.956589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.957382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:18.959145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:18.986683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 21/40] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:18.987986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b96e9e-faa6-420d-9ac8-2950d41b520f None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:18.988277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b96e9e-faa6-420d-9ac8-2950d41b520f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:18.993589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b96e9e-faa6-420d-9ac8-2950d41b520f None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:18.993589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 22/41] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:19.006422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:19.006803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:19.007207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:19.028032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:19.029330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:19.074521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:19.091726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 20/42] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:06:18 2022] POST /identity/v3/projects => generated 301 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:19.094202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:19.111963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 23/43] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:19.136017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:19.183052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:19.225715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb378c-1ed0-474f-a514-67aba31a8a0b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:19.260120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00c431-7e53-4adb-862c-ce995db8df0b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:19.269497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd6038-3291-4c15-ac9e-bb422e5ec8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:19.270249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd6038-3291-4c15-ac9e-bb422e5ec8f8 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:19.270606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd6038-3291-4c15-ac9e-bb422e5ec8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:19.270953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd6038-3291-4c15-ac9e-bb422e5ec8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:19.271791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd6038-3291-4c15-ac9e-bb422e5ec8f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:19.274587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd6038-3291-4c15-ac9e-bb422e5ec8f8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:19.300551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:19.301342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:19.301699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:19.301984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:19.303185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:19.304823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:19.306455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 24/44] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:19.312807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:19.359626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 21/45] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 10 13:06:19 2022] POST /identity/v3/roles => generated 223 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:19.371127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:19.435168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b33aae-9d76-4097-b65b-462fc80ae7c6 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:19.473159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f93956-f2bf-4625-8430-b1599c782f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:19.473798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f93956-f2bf-4625-8430-b1599c782f60 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:19.474090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f93956-f2bf-4625-8430-b1599c782f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:19.474359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f93956-f2bf-4625-8430-b1599c782f60 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:19.475023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f93956-f2bf-4625-8430-b1599c782f60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:19.476523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f93956-f2bf-4625-8430-b1599c782f60 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:19.566646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 25/46] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:06:19 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:06:20.726819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac99626-fdcf-4568-8e7a-6003d2219897 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:20.727354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac99626-fdcf-4568-8e7a-6003d2219897 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:20.727449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac99626-fdcf-4568-8e7a-6003d2219897 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:20.728731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 22/47] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:20.733943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:20.734339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:20.734567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:20.753715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:20.754890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:20.821121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 26/48] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:20.843496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac99626-fdcf-4568-8e7a-6003d2219897 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:20.942909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac99626-fdcf-4568-8e7a-6003d2219897 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:20.967259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:20.975173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:20.975804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:20.976086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:20.976362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:20.987188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:20.991922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:20.992734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:20.993517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 23/49] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:20.998836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:21.024237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:21.099405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeefcf4-3ad0-4de8-8b7f-48cc01bc25b4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:21.133250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:21.133860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.134144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.134419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.135092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:21.135397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:21.136780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:21.145032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 27/50] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:21.150365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:21.247921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2ecae-9394-42ca-bf84-2308de359e2c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:21.280796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:21.281456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.281738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.282012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.283904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:21.285376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:21.286945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 24/51] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:21 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:21.292317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:21.377410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:21.391438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c9a91-1839-4a0a-8c14-fe51d5bbe10a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:21.425325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d986b-b3b5-4fa4-8bf8-db33ee9c2b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:21.425957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d986b-b3b5-4fa4-8bf8-db33ee9c2b44 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.426243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d986b-b3b5-4fa4-8bf8-db33ee9c2b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.426522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d986b-b3b5-4fa4-8bf8-db33ee9c2b44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.427630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d986b-b3b5-4fa4-8bf8-db33ee9c2b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:21.429095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d986b-b3b5-4fa4-8bf8-db33ee9c2b44 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:21.430808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 28/52] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:21.436514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:21.491592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:21.535333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655043f-8792-467e-af78-5f0fea140727 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:21.570831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:21.571438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.571722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.571998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.572724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.domain.id=default, user.id=40886c602f2b4c218564e5847aed724b) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:21.573062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:21.574481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88eba-e047-4dba-9b9b-6c6c66a69864 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:21.582466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 25/53] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Aug 10 13:06:21 2022] GET /identity/v3/role_assignments?user.id=40886c602f2b4c218564e5847aed724b&scope.domain.id=default&role.id=dd76871a868143b482c4982027fdc1f5 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:21.592048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391ff4c-cc81-4f4a-ac9f-79014f13829a None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.592336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391ff4c-cc81-4f4a-ac9f-79014f13829a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.592615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391ff4c-cc81-4f4a-ac9f-79014f13829a None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.594029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 29/54] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:21.601122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.601420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.601703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.621387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:21.622638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:21.698782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 26/55] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:21.722820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391ff4c-cc81-4f4a-ac9f-79014f13829a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:21.825186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391ff4c-cc81-4f4a-ac9f-79014f13829a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:21.858180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024bd2f-2ac6-4967-a81a-a02c0aae6fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:21.858808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024bd2f-2ac6-4967-a81a-a02c0aae6fed None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.859094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024bd2f-2ac6-4967-a81a-a02c0aae6fed None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.859373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024bd2f-2ac6-4967-a81a-a02c0aae6fed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.860535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024bd2f-2ac6-4967-a81a-a02c0aae6fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:21.862018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024bd2f-2ac6-4967-a81a-a02c0aae6fed None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:21.863581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 30/56] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:21.869356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:21.917831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:21.965001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d3d7-4c71-4df1-a83e-a94971e83c3b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:21.997903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475ca97-bac9-47da-8bc4-d8235d5f82d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:21.998501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475ca97-bac9-47da-8bc4-d8235d5f82d4 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:21.998787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475ca97-bac9-47da-8bc4-d8235d5f82d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:21.999063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475ca97-bac9-47da-8bc4-d8235d5f82d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:21.999709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475ca97-bac9-47da-8bc4-d8235d5f82d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:22.001793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475ca97-bac9-47da-8bc4-d8235d5f82d4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:22.078938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 27/57] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Aug 10 13:06:21 2022] POST /identity/v3/projects => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:24.137682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671edc9-fdda-4b38-8483-d29d7b7e58fd None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:24.138190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671edc9-fdda-4b38-8483-d29d7b7e58fd None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:24.138271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671edc9-fdda-4b38-8483-d29d7b7e58fd None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:24.139567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 31/58] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:24.144864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:24.145289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:24.145455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:24.163437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:24.164620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:24.230431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 28/59] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:24.254388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671edc9-fdda-4b38-8483-d29d7b7e58fd None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:24.278986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:24.353490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671edc9-fdda-4b38-8483-d29d7b7e58fd None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:24.384927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244b46f-02f8-4ff8-8888-4957ffb06a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:24.385578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244b46f-02f8-4ff8-8888-4957ffb06a54 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:24.385871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244b46f-02f8-4ff8-8888-4957ffb06a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:24.386138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244b46f-02f8-4ff8-8888-4957ffb06a54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:24.387244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244b46f-02f8-4ff8-8888-4957ffb06a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:24.388737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244b46f-02f8-4ff8-8888-4957ffb06a54 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:24.390266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 32/60] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:24.396317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:24.426530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:24.493497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa8172-4ac3-446c-8fbb-918145999579 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:24.527095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8bed7-054b-45fd-b446-91aed1507582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:24.527691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8bed7-054b-45fd-b446-91aed1507582 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:24.527974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8bed7-054b-45fd-b446-91aed1507582 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:24.528250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8bed7-054b-45fd-b446-91aed1507582 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:24.528890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8bed7-054b-45fd-b446-91aed1507582 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:24.530453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8bed7-054b-45fd-b446-91aed1507582 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:24.626938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 29/61] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:24 2022] POST /identity/v3/users => generated 275 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:27.717137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a79e7f-db7e-4b09-953c-49cc7167da4b None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:27.718087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a79e7f-db7e-4b09-953c-49cc7167da4b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:27.718583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a79e7f-db7e-4b09-953c-49cc7167da4b None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:27.721137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 33/62] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:27.726045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:27.726902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:27.727377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:27.746598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:27.748146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:27.789245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e661c5-eaa1-4228-ae68-2f9e76a74f1b None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:27.789868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e661c5-eaa1-4228-ae68-2f9e76a74f1b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:27.790367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e661c5-eaa1-4228-ae68-2f9e76a74f1b None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:27.791928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 34/63] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:27.799542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d969a99-7877-4738-84e4-4145169b5add None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:27.800135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d969a99-7877-4738-84e4-4145169b5add None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:27.800613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d969a99-7877-4738-84e4-4145169b5add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:27.825998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9d969a99-7877-4738-84e4-4145169b5add None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:27.833112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9d969a99-7877-4738-84e4-4145169b5add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:27.883909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 30/64] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:27.908697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:27.927668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:27.933733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 35/65] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:27.935354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090f3ea-0bc6-40d5-9426-467af881d1f2 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:27.935862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090f3ea-0bc6-40d5-9426-467af881d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:27.936370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090f3ea-0bc6-40d5-9426-467af881d1f2 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:27.938170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 36/66] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:27.950330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cfe35-65e4-496c-925b-235837640754 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:27.950631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cfe35-65e4-496c-925b-235837640754 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:27.950897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cfe35-65e4-496c-925b-235837640754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:27.978204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-706cfe35-65e4-496c-925b-235837640754 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:27.979410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-706cfe35-65e4-496c-925b-235837640754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:28.001718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.032188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27232729-cf0e-4cd5-959f-d98b3219ba36 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.071293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.071964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.072310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.072591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.073855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 37/67] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:28.074989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cfe35-65e4-496c-925b-235837640754 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.097911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:28.107329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:28.108645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:28.110725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 31/68] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:28.114816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.204380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.207858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cfe35-65e4-496c-925b-235837640754 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.237687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9982c-9069-42d4-a223-a427ad52ebe2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.258163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.258844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.259132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.259517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.271144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:28.276575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:28.278679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 38/69] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:28.280702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.282954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.282954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.282954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.282954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.293974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:28.297240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:28.300366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:28.301236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:28.302009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:28.302569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 32/70] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:28.303100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.393011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd655f-b66c-4733-a7a6-e42fd47082c1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.413965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4466a66-ff48-4bec-bbc3-d23d18f77320 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.429918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.430527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.430814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.431091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.431771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:28.432078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:28.433520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6aa8-cf86-46ba-8868-76c2ec3e1ec6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:28.446663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 39/71] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:28.446663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af0925-443b-427a-a656-41cde843b0b8 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.446663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af0925-443b-427a-a656-41cde843b0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.446663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af0925-443b-427a-a656-41cde843b0b8 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.446663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 40/72] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:28.446663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af0925-443b-427a-a656-41cde843b0b8 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.456160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.456733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.457034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.457313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.457990 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:28.458497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:28.459994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:28.469274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.474583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 33/73] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:28.479502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.527355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.558789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af0925-443b-427a-a656-41cde843b0b8 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.590224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f0e88-edda-448a-8549-df1beaf2c030 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.593715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.594326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.594690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.594968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.595771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:28.596078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:28.597496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637db9c-9769-49be-ad7d-d7df7836c7bc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:28.605383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 41/74] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:28.606566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dc283-ca60-4312-8916-ea22031ce126 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.606855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dc283-ca60-4312-8916-ea22031ce126 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.607129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dc283-ca60-4312-8916-ea22031ce126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.624564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e68dc283-ca60-4312-8916-ea22031ce126 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:28.626416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e68dc283-ca60-4312-8916-ea22031ce126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:28.628612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.629282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.629565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.629840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.644558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:28.646078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:28.647999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 34/75] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:28.648789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.697070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 42/76] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:28.698313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0618ca-79e7-4171-ac01-8d65f47c1cb3 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.698603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0618ca-79e7-4171-ac01-8d65f47c1cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.698881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0618ca-79e7-4171-ac01-8d65f47c1cb3 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.700110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 43/77] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:28.705198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0618ca-79e7-4171-ac01-8d65f47c1cb3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.709987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.749039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c18e-c4f6-4d33-98e6-1cb62fe6ac59 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.777095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.783989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.784707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.784948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.785303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.786435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:28.787911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:28.789521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 35/78] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:28.793793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.816301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0618ca-79e7-4171-ac01-8d65f47c1cb3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.849355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1787d-d2d0-4500-bb91-99c90c78d915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.849982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1787d-d2d0-4500-bb91-99c90c78d915 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.850262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1787d-d2d0-4500-bb91-99c90c78d915 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.850539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1787d-d2d0-4500-bb91-99c90c78d915 None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.851654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1787d-d2d0-4500-bb91-99c90c78d915 None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:28.853131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1787d-d2d0-4500-bb91-99c90c78d915 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:28.854916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 44/79] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:28.855831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.856116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.856388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.873222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:28.874404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:28.894436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbeb00-9ff7-485e-a5a3-de576925db54 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:28.919441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:28.928603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:28.929243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:28.929524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:28.929793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:28.941506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 45/80] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:28.942158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:28.999628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.001654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.007907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.046265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9d3f9-b4e6-45eb-ba32-52ef9f226905 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.078813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.080009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.080289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.080561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.090306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:29.095479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:29.096415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:29.097234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 46/81] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:29.097883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.104026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 36/82] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:29.105124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.132842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.195383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28dd1c-b30f-463a-a909-cfe003456b7d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.208400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d313b-483c-435e-9425-bf927a17a037 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.217193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.228930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.229564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.229859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.230137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.242691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.243332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.243617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.243892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.302356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.303848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.310964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.312414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.338493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.401774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 47/83] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:29.401774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.410981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 37/84] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:29.413968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.429875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.510685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506c19d-6a97-4553-8df4-9e666a1a7f51 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.510874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652b8d-e095-4ea4-b39d-d80cfbed06e5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.536024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.544136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.549121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.549237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.549237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.549383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.549459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.549549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.549549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.549688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:29.549688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.549810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.549882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.549946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.550600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 48/85] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:29.551358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.557359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 38/86] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:29.558046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.640370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.650279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0d127-fd51-472c-b522-243ee99988e6 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.657080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6d69b-1f0e-4fa9-83aa-b7391010a48b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.686083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.686729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.687019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.687300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.689027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.689978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.690559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.690653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.690878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.691155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.692210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 49/87] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:29.692326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.693020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.693832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.700083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 39/88] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:29.700083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.754953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.794358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251b395-57fc-4ebc-bc73-84b451c149a9 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.800964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694f59-af86-430d-affb-4d132a2f53a4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.832377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.832939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.833249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.833522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.834647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.836117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.837248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.837880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 50/89] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:29.837979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.838122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.838387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.838551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.839021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.840438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.866689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:29.918313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 40/90] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:29 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:29.919144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:29.942267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a036dc-66d2-49aa-9abc-776f86819e74 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:29.981925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:29.982580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:29.982868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:29.983143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:29.984346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:29.985867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:29.992196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 51/91] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:29.992196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:30.038550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909243-a1c0-45f5-9f63-217793afe80b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:30.053451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:30.077349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:30.078557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:30.078557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:30.079498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:30.080589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=a8045692cd1e4d48be5039398bca0cc4) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:30.082289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:30.084562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:30.092228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 41/92] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/role_assignments?user.id=a8045692cd1e4d48be5039398bca0cc4&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:30.092898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:30.100609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:30.108764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfaf5f-7bc8-432c-a30f-4f27883358d5 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:30.166125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:30.166856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:30.167140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:30.167519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:30.169325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=40886c602f2b4c218564e5847aed724b) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:30.169639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:30.171259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:30.179419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 52/93] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:29 2022] GET /identity/v3/role_assignments?user.id=40886c602f2b4c218564e5847aed724b&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:30.180381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:30.271738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb1d5-3989-4d08-b293-b2800d3ba564 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:30.324783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:30.324783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:30.324783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:30.326963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:30.327152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] RBAC: Adding query filter params (role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=a8045692cd1e4d48be5039398bca0cc4) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:30.327462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:30.332018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:30.337875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 42/94] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:30 2022] GET /identity/v3/role_assignments?user.id=a8045692cd1e4d48be5039398bca0cc4&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:30.355342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff822a4c-a1ab-483d-8ba8-9511f4fcc7d7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:30.401597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:30.402545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:30.402967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:30.403380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:30.440540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:30.507084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:30.508587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:30.638066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 53/95] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:30.644131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:30.731041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:30.809891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef6516-fd10-4b4b-8a6b-c2e7bbc1d609 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:30.856901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9252584-e2db-49a9-bd26-d15bda6531d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:30.857634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9252584-e2db-49a9-bd26-d15bda6531d9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:30.857921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9252584-e2db-49a9-bd26-d15bda6531d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:30.858198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9252584-e2db-49a9-bd26-d15bda6531d9 None admin] PATH_INFO: `/v3/projects/a1389069273341eaa0c38004e21d5456` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:30.859330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9252584-e2db-49a9-bd26-d15bda6531d9 None admin] RBAC: Authorizing `identity:get_project(project_id=a1389069273341eaa0c38004e21d5456)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:30.860843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9252584-e2db-49a9-bd26-d15bda6531d9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:30.862867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 43/96] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:30 2022] GET /identity/v3/projects/a1389069273341eaa0c38004e21d5456 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:30.870279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:30.939631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:30.988265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e06fe-58c0-4dec-b54d-7be512f02bae None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:31.031394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:31.032652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:31.032984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:31.033240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:31.033964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=a1389069273341eaa0c38004e21d5456, user.id=a8045692cd1e4d48be5039398bca0cc4) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:31.034272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:31.035725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57d4d-1204-4620-a4e0-040a05b86ea5 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:31.045532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 54/97] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:30 2022] GET /identity/v3/role_assignments?user.id=a8045692cd1e4d48be5039398bca0cc4&scope.project.id=a1389069273341eaa0c38004e21d5456&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:32.629529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048663-6cae-4a02-9b60-254bfd9fcb30 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:32.630039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048663-6cae-4a02-9b60-254bfd9fcb30 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:32.630039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048663-6cae-4a02-9b60-254bfd9fcb30 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:32.631698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 44/98] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:32.636629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee7751-f04a-489a-8a21-42c9b50bab98 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:32.637047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee7751-f04a-489a-8a21-42c9b50bab98 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:32.637232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee7751-f04a-489a-8a21-42c9b50bab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:32.664447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-51ee7751-f04a-489a-8a21-42c9b50bab98 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:32.665675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-51ee7751-f04a-489a-8a21-42c9b50bab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:32.763591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 55/99] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:32.786587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74048663-6cae-4a02-9b60-254bfd9fcb30 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:32.907535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74048663-6cae-4a02-9b60-254bfd9fcb30 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:32.919476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:32.949724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:32.950365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:32.950649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:32.950925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:32.955649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbb565-02c3-4365-8186-ae4d24a2be40 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:32.955938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbb565-02c3-4365-8186-ae4d24a2be40 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:32.956216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbb565-02c3-4365-8186-ae4d24a2be40 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:32.957779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 56/100] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:32.962383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:32.962669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:32.963091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:32.979529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:32.986938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:32.987791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:32.988536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 45/101] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:32.991479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:32.992771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:32.997218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.081055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:33.117151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 57/102] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:33.145576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.158775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f438b-48f6-402b-8507-fce598a05ede None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:33.208948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:33.209585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.209869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.210145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.210820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:33.211125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:33.212504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:33.229535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 46/103] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:33.238864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.278840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:33.348922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb114e-8315-482b-b942-5f30e73b6a34 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:33.376150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474aad4-fc8b-4dcd-9abb-7cb6868c2208 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:33.421371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:33.421945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.422235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.422547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.437185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6b62-46c9-4ac1-a77b-ff7f9dc4216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:33.437832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6b62-46c9-4ac1-a77b-ff7f9dc4216b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.438642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6b62-46c9-4ac1-a77b-ff7f9dc4216b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.438642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6b62-46c9-4ac1-a77b-ff7f9dc4216b None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.442116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6b62-46c9-4ac1-a77b-ff7f9dc4216b None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:33.443593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6b62-46c9-4ac1-a77b-ff7f9dc4216b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:33.445281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 47/104] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:33 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:33.446488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfe8d6-629f-45c3-91e9-e45ef6ad37e3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.447380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:33.452248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:33.461606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 58/105] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:33.462352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.463038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfe8d6-629f-45c3-91e9-e45ef6ad37e3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.463038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfe8d6-629f-45c3-91e9-e45ef6ad37e3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.463038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 48/106] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:33.463038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfe8d6-629f-45c3-91e9-e45ef6ad37e3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.545390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:33.666454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950addb9-e563-4b84-bb5a-026aee20ba9b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:33.709443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:33.716687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfe8d6-629f-45c3-91e9-e45ef6ad37e3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:33.731232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c144640-c628-46c8-931f-5347a41b629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:33.732263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c144640-c628-46c8-931f-5347a41b629b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.732800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c144640-c628-46c8-931f-5347a41b629b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.733315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c144640-c628-46c8-931f-5347a41b629b None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.734667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c144640-c628-46c8-931f-5347a41b629b None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:33.736415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c144640-c628-46c8-931f-5347a41b629b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:33.738350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 59/107] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:33 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:33.739762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.740265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.740755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.760350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:33.761360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:33.762044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.762334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:33.762436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.762719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.763446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:33.763817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:33.766891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d563-9f06-4b12-a824-6f340961ba53 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:33.775606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 49/108] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:33.777361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05dc8-e800-41eb-9310-f9e5d6a80d42 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.777717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05dc8-e800-41eb-9310-f9e5d6a80d42 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.778061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05dc8-e800-41eb-9310-f9e5d6a80d42 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.779799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 50/109] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:33.784890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b05dc8-e800-41eb-9310-f9e5d6a80d42 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.856495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 60/110] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:33.857664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:33.903599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b05dc8-e800-41eb-9310-f9e5d6a80d42 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:33.916200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:33.942762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927939a-e367-49e2-b6e1-d6e710de0c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:33.943429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927939a-e367-49e2-b6e1-d6e710de0c9a None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:33.951925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927939a-e367-49e2-b6e1-d6e710de0c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:33.951925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927939a-e367-49e2-b6e1-d6e710de0c9a None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a/users/a8045692cd1e4d48be5039398bca0cc4/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:33.951925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927939a-e367-49e2-b6e1-d6e710de0c9a None admin] RBAC: Authorizing `identity:create_grant(project_id=937d691e04a147e0a4028b930e05179a, user_id=a8045692cd1e4d48be5039398bca0cc4, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:33.951925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927939a-e367-49e2-b6e1-d6e710de0c9a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:33.972887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2d9a7-52d6-433c-a20d-f1769d81ff5e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.006702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0e62d-00fd-458c-8cd9-de01b293daa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.007420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0e62d-00fd-458c-8cd9-de01b293daa4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.007702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0e62d-00fd-458c-8cd9-de01b293daa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.007978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0e62d-00fd-458c-8cd9-de01b293daa4 None admin] PATH_INFO: `/v3/users/47d9f41513624d38a8773a1b23aeffd7` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.021173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0e62d-00fd-458c-8cd9-de01b293daa4 None admin] RBAC: Authorizing `identity:get_user(user_id=47d9f41513624d38a8773a1b23aeffd7)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:34.022619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0e62d-00fd-458c-8cd9-de01b293daa4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:34.024268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 61/111] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:33 2022] GET /identity/v3/users/47d9f41513624d38a8773a1b23aeffd7 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:34.025396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98d291-766d-4fc4-a341-5501911a00b1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.025683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98d291-766d-4fc4-a341-5501911a00b1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.025959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98d291-766d-4fc4-a341-5501911a00b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.036490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 51/112] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:33 2022] PUT /identity/v3/projects/937d691e04a147e0a4028b930e05179a/users/a8045692cd1e4d48be5039398bca0cc4/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:34.043946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1f98d291-766d-4fc4-a341-5501911a00b1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:34.045216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1f98d291-766d-4fc4-a341-5501911a00b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:34.054487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316dc8b-0dbe-4054-ac4b-4c3abf3e783c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.054772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316dc8b-0dbe-4054-ac4b-4c3abf3e783c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.055051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316dc8b-0dbe-4054-ac4b-4c3abf3e783c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.056421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 52/113] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:34 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:34.057723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316dc8b-0dbe-4054-ac4b-4c3abf3e783c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.083021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.136034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 62/114] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:34.136034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4ec2-5397-42d4-885e-bb3ee668f224 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.136034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4ec2-5397-42d4-885e-bb3ee668f224 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.136034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4ec2-5397-42d4-885e-bb3ee668f224 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.136034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 63/115] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:34.137299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4ec2-5397-42d4-885e-bb3ee668f224 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.170111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.193879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316dc8b-0dbe-4054-ac4b-4c3abf3e783c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.231134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe551a40-e4ba-44a4-a0dc-591b237a8c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.231692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe551a40-e4ba-44a4-a0dc-591b237a8c59 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.231968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe551a40-e4ba-44a4-a0dc-591b237a8c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.232238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe551a40-e4ba-44a4-a0dc-591b237a8c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.243657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe551a40-e4ba-44a4-a0dc-591b237a8c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-fe551a40-e4ba-44a4-a0dc-591b237a8c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:34.248967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:34.249846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:34.251188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 53/116] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:34.251902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcb90-817a-4072-8f50-5d6013f21539 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.252948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcb90-817a-4072-8f50-5d6013f21539 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.252948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcb90-817a-4072-8f50-5d6013f21539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.267896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4ec2-5397-42d4-885e-bb3ee668f224 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.270480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0edbcb90-817a-4072-8f50-5d6013f21539 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:34.271680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0edbcb90-817a-4072-8f50-5d6013f21539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:34.306733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.307499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.307788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.308060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.359357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 54/117] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:34.359357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.359357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.359357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.373022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:34.374249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:34.377873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.394369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:34.395956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:34.423993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.448867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 55/118] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:34.449835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.516325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 64/119] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:34.516325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.569780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e571153-75a5-4d47-9bf9-58410dd1c31a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.605489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.606429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.606906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.607347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.615284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.618070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:34.622887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:34.623738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:34.624511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:34.626034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b596-6cbf-48cd-9ca6-a6f865ab5d4e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.626106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 56/120] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:34.631075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.655597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.665046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.665657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.665947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.666223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.666890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:34.667388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:34.668579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:34.678849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 65/121] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:34.679959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.754515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc2249-048d-4f7c-ac0f-5c7ebae4754d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.786046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994edf9-02e9-4ff5-ac9c-70693d544312 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.792128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.792772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.793305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.793482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.794625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:34.796100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:34.803380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 57/122] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:34.803380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.830042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.831033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.831515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.831987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.842900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:34.849143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:34.850225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:34.851159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 66/123] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:34.851926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.885212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.912528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:34.919619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8cd7-03c0-4120-bef6-28f260470176 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:34.954612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:34.955240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:34.955524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:34.955797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:34.956929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:34.958420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:34.964645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 58/124] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:34.964645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:34.974890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c514dd-5467-40c8-ab0d-e158780fbfe0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.011111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.011709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.011994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.012269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.012980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:35.013266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.014644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.027583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 67/125] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.027583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.071810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4210-b62b-4a89-a0a3-203b1177f2b2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.105587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.106227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.106514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.106791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.107921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.109393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.111090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 59/126] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:34 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.111866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.116595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:35.138075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559aa6-5a09-40c1-b424-1ed10e528130 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.157422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:35.179564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.180184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.180474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.180754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.181428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.182915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.219968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266bf25-b8c3-4bc0-a4f2-e69a4e10003e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.252892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.253525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.253807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.254077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.254778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:35.255075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.256431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.264467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 60/127] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.265335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.275463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 68/128] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:06:35 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:35.276623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.365091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:35.405832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ca68-30fa-4034-92a3-95d07957cbd9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.411057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b22cb-a01b-4bc4-beb7-5e30634a4d8d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.448632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.449469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.449851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.450363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.451229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.451661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.451841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.452243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.453165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] RBAC: Adding query filter params (role.id=3e730d996bc3433bba5409328aabef27, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=47d9f41513624d38a8773a1b23aeffd7) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:35.453511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.454310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.455916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.456032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 69/129] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.456926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.466475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 61/130] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/role_assignments?user.id=47d9f41513624d38a8773a1b23aeffd7&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=3e730d996bc3433bba5409328aabef27 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.468018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.495888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:35.588588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40870e27-8383-4c4a-a59a-6d70f0dbf55c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.611349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16f6bc-6673-4a96-9360-3a0c17bbb987 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.632955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.634057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.634611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.635086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.636501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.638376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.640392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 70/131] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.641563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.648547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:35.655878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.656527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.656836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.657648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.660558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.662470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.663874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 62/132] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.664635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.769166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:35.773329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d60b-5f90-42dc-b666-79427281f373 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.811881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49da41e-f124-4e64-bd10-cc64cc69172b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:35.850277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.850931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.851219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.851424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:35.851538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.852190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:35.852482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:35.852761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a/users/40886c602f2b4c218564e5847aed724b/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:35.854815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] RBAC: Authorizing `identity:create_grant(project_id=937d691e04a147e0a4028b930e05179a, user_id=40886c602f2b4c218564e5847aed724b, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.856321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.857688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:35.859157 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:35.862783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 71/133] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:35.862783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:35.958462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 63/134] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:35 2022] PUT /identity/v3/projects/937d691e04a147e0a4028b930e05179a/users/40886c602f2b4c218564e5847aed724b/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:35.958462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:36.021539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41cf7e-d269-479b-9e2e-2cdf903abf36 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:36.097828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:36.098730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.099324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.099738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] PATH_INFO: `/v3/projects/a1389069273341eaa0c38004e21d5456` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.101568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] RBAC: Authorizing `identity:get_project(project_id=a1389069273341eaa0c38004e21d5456)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:36.103656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:36.104245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:36.106707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 72/135] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:35 2022] GET /identity/v3/projects/a1389069273341eaa0c38004e21d5456 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:36.113798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:36.177979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:36.203239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48495395-d6f2-48b0-8387-3832b0ef0e66 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:36.250425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e3053-285a-4bf6-b51d-9b9a4c9fe307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:36.251849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e3053-285a-4bf6-b51d-9b9a4c9fe307 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.252238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e3053-285a-4bf6-b51d-9b9a4c9fe307 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.252918 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e3053-285a-4bf6-b51d-9b9a4c9fe307 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a/users/a8045692cd1e4d48be5039398bca0cc4/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.257248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e3053-285a-4bf6-b51d-9b9a4c9fe307 None admin] RBAC: Authorizing `identity:create_grant(project_id=937d691e04a147e0a4028b930e05179a, user_id=a8045692cd1e4d48be5039398bca0cc4, role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:36.260540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e3053-285a-4bf6-b51d-9b9a4c9fe307 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:36.260657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fb09f-fa12-49e9-b17f-afb9f52edb9b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:36.308369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:36.309077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.309357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.309635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] PATH_INFO: `/v3/projects/a1389069273341eaa0c38004e21d5456/users/a8045692cd1e4d48be5039398bca0cc4/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.311625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1389069273341eaa0c38004e21d5456, user_id=a8045692cd1e4d48be5039398bca0cc4, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:36.313968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:36.349196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 64/136] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:35 2022] PUT /identity/v3/projects/937d691e04a147e0a4028b930e05179a/users/a8045692cd1e4d48be5039398bca0cc4/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:36.350597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb2a22-d144-4a26-bca5-a785d18150d7 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.350989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb2a22-d144-4a26-bca5-a785d18150d7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.351390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb2a22-d144-4a26-bca5-a785d18150d7 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.353073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 65/137] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:36.358464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.358887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.359271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.379846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:36.381407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:36.386614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 73/138] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:36 2022] PUT /identity/v3/projects/a1389069273341eaa0c38004e21d5456/users/a8045692cd1e4d48be5039398bca0cc4/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:06:36.481410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 66/139] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:36.516525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:36.626377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:36.632779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41c9d5-7ef3-40f1-8395-49c507b5b345 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:36.670795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:36.671467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.671985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.672486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.685330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:36.697801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:36.698657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:36.699450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 74/140] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:36.701972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:36.769167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:36.825836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa7c75-2f2d-4cdf-bdbd-9205f6a04818 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:36.866535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:36.867380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:36.867800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:36.868209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:36.869052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:36.869492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:36.870999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:36.882324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 67/141] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:36.885166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:37.005021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82e6e4-0747-46ee-afee-8aa8201ccb1c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:37.054674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:37.055533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:37.056006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:37.056412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:37.057726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:37.059330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:37.061063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 75/142] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:36 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:37.066922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:37.176922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5be8-a186-45c0-8ba0-243679fb6a31 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:37.194132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:37.215541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:37.216157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:37.216442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:37.216714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:37.286203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:37.287676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:37.389488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 68/143] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:06:37.395537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:37.440166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:37.502211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbe6c6-7003-40ed-ac2f-84bd5528c835 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:37.537029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92173a00-4540-42d8-afef-5f6be407581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:37.537694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92173a00-4540-42d8-afef-5f6be407581d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:37.538297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92173a00-4540-42d8-afef-5f6be407581d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:37.538648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92173a00-4540-42d8-afef-5f6be407581d None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:37.540538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92173a00-4540-42d8-afef-5f6be407581d None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:37.542092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92173a00-4540-42d8-afef-5f6be407581d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:37.548800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 76/144] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:37 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:37.550174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:37.565353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:37.650504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0818d25-25b1-41e0-8e86-354509283e4f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:37.686122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:37.686736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:37.687020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:37.687300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:37.688054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=a8045692cd1e4d48be5039398bca0cc4) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:37.688485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:37.689994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca14344-64f1-4410-81fc-7d6c487f746f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:37.696041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 69/145] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:37 2022] GET /identity/v3/role_assignments?user.id=a8045692cd1e4d48be5039398bca0cc4&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:38.294570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330a1d-53c6-47db-b497-e2865bd86f40 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.295084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330a1d-53c6-47db-b497-e2865bd86f40 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.295173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330a1d-53c6-47db-b497-e2865bd86f40 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.296725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 77/146] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:38.302342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccec72-5e96-46e5-bb16-397f2e9d3441 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.302764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccec72-5e96-46e5-bb16-397f2e9d3441 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.302936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccec72-5e96-46e5-bb16-397f2e9d3441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.337850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b2ccec72-5e96-46e5-bb16-397f2e9d3441 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:38.339051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b2ccec72-5e96-46e5-bb16-397f2e9d3441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:38.416420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 70/147] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:38.444425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5330a1d-53c6-47db-b497-e2865bd86f40 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:38.525455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd034-43dc-40a2-9a64-541ed7eaa420 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.526144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd034-43dc-40a2-9a64-541ed7eaa420 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.526820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd034-43dc-40a2-9a64-541ed7eaa420 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.528370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 71/148] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:38.533872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.534214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.534553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.554742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:38.556641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:38.565811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:38.576842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:38.596338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5330a1d-53c6-47db-b497-e2865bd86f40 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:38.657741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 72/149] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:38.674768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:38.675419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.675701 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.675983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.685214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:38.700365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:38.705699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:38.706584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:38.707383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:38.708480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 78/150] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:38.716495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:38.863809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78b793-5fed-4201-ab41-16cf96d61bd9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:38.905992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b5cd0-d8d4-4086-b95b-fa52331d57d9 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:38.925880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:38.926520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.926793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.927058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.928150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:38.930377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:38.931941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 73/151] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:38.940498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:38.945295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:38.979552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:38.980252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:38.980533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:38.980809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:38.981509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:38.981809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:38.983191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:38.995246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 79/152] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:39.004051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.025924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:39.122771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddad2c-4622-4a50-8cb1-f73ee194074c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.177381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0041f-2228-4344-a974-784d76d55a8b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.180728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68539731-3c5e-4cc6-aa50-634313af77b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.181216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539731-3c5e-4cc6-aa50-634313af77b2 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.181495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539731-3c5e-4cc6-aa50-634313af77b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.181765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539731-3c5e-4cc6-aa50-634313af77b2 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.182472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68539731-3c5e-4cc6-aa50-634313af77b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:39.183962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68539731-3c5e-4cc6-aa50-634313af77b2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:39.223535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.224172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.224450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.224719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] PATH_INFO: `/v3/users/47d9f41513624d38a8773a1b23aeffd7` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.226038 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] RBAC: Authorizing `identity:get_user(user_id=47d9f41513624d38a8773a1b23aeffd7)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:39.227480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:39.229337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 80/153] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:39 2022] GET /identity/v3/users/47d9f41513624d38a8773a1b23aeffd7 => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:39.234773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.296024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 74/154] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:38 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:39.297749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294b567-173e-4ae2-8def-6b4af481b8eb None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.298123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294b567-173e-4ae2-8def-6b4af481b8eb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.298473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294b567-173e-4ae2-8def-6b4af481b8eb None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.299866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 75/155] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:39.306336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53217c9-de2c-45fb-968c-680ad399c89f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.306868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53217c9-de2c-45fb-968c-680ad399c89f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.307271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53217c9-de2c-45fb-968c-680ad399c89f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.312942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 76/156] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:39.312942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3993ad-88db-4bcf-9770-39187852cffa None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.312942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3993ad-88db-4bcf-9770-39187852cffa None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.312942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3993ad-88db-4bcf-9770-39187852cffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.332010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2a3993ad-88db-4bcf-9770-39187852cffa None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:39.333504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a3993ad-88db-4bcf-9770-39187852cffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:39.338822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:39.360279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62a23f-af5c-49f2-8568-fbdf5b86dfd7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.393557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:39.405056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.406027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.406541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.407727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.409525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:39.414377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 77/157] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:39.414514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:39.414514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 81/158] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:39 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:39.418854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.419342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.419777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.423334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.449999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:39.451432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:39.537308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 78/159] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:39.541295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.590280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563429c9-97ee-4fed-ae8a-96240a5a2630 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.633192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.634115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.634403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.634729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a/users/47d9f41513624d38a8773a1b23aeffd7/roles/3e730d996bc3433bba5409328aabef27` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.637630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] RBAC: Authorizing `identity:create_grant(project_id=937d691e04a147e0a4028b930e05179a, user_id=47d9f41513624d38a8773a1b23aeffd7, role_id=3e730d996bc3433bba5409328aabef27)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:39.639106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:39.659046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc4752-df68-478e-95af-1df1c7aa8964 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.678119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:39.695559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2103-a2dd-4cd5-81e5-4c7e93758a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.696202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2103-a2dd-4cd5-81e5-4c7e93758a29 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.696487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2103-a2dd-4cd5-81e5-4c7e93758a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.696760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2103-a2dd-4cd5-81e5-4c7e93758a29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.707623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c2103-a2dd-4cd5-81e5-4c7e93758a29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0e3c2103-a2dd-4cd5-81e5-4c7e93758a29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:39.712434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:39.713370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:39.714224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 79/160] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:39.715422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85c62-d4d6-4295-810f-682ef4ca56ce None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.715704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85c62-d4d6-4295-810f-682ef4ca56ce None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.715976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85c62-d4d6-4295-810f-682ef4ca56ce None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.716574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 82/161] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:39 2022] PUT /identity/v3/projects/937d691e04a147e0a4028b930e05179a/users/47d9f41513624d38a8773a1b23aeffd7/roles/3e730d996bc3433bba5409328aabef27 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:06:39.719127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 80/162] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:39.719127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85c62-d4d6-4295-810f-682ef4ca56ce None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.727500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.752700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:39.863416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85c62-d4d6-4295-810f-682ef4ca56ce None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.878107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84d616-5eb6-418d-a27a-ca7ff1defbae None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:39.906380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f716f-ddde-4517-b403-1129027c030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.908337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f716f-ddde-4517-b403-1129027c030a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.908672 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f716f-ddde-4517-b403-1129027c030a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.909035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f716f-ddde-4517-b403-1129027c030a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.924576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f716f-ddde-4517-b403-1129027c030a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:39.926185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:39.926814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.927099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.927388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.928065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:39.928368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6f7f716f-ddde-4517-b403-1129027c030a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:39.929533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:39.930399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:39.931161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:39.931776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:39.931844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 81/163] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:39.932368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:39.932368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:39.933055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:39.943706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 83/164] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:39.946339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:39.966022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:39.966316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:40.004621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:40.046160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 82/165] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:40.047084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.052533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:40.091234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b0cd-9a79-4e29-9337-e029d5d307a2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:40.138618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:40.139558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:40.139854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:40.140400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:40.141660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:40.141979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:40.143957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:40.155473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 84/166] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:40.156555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.162895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62467c-be8e-40dc-b816-07ff3a2d2e2c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:40.199399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:40.200113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:40.200391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:40.200663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] PATH_INFO: `/v3/users/40886c602f2b4c218564e5847aed724b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:40.202363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] RBAC: Authorizing `identity:get_user(user_id=40886c602f2b4c218564e5847aed724b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:40.204164 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:40.206092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 83/167] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/users/40886c602f2b4c218564e5847aed724b => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:40.206871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.256546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:40.315798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2b1d-4b96-4872-adc5-5c33b5864dd4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:40.362446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385248f-9352-48b8-9c7c-f4f50193ecb5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:40.380517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:40.381189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:40.381460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:40.383126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:40.387797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:40.402124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:40.408326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:40.409286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:40.410079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 85/168] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:40.411307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.437877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:40.439020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:40.439303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:40.439573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:40.441686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:40.443142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:40.449087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 84/169] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:40.449087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.561139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bbac0-5093-491b-8938-87bb4c226f36 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:40.609328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:40.610434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:40.610944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:40.611411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:40.697683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:40.706319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:40.710205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:40.742972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a5c1-e11d-4ee0-abea-67f5bdd3b7b0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:40.745358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:40.830005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:40.830637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:40.830923 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:40.831200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:40.831889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:40.832198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:40.833638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:40.847245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 85/170] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:40.848088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.873194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 86/171] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:40.873990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:40.987221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074096b-69bd-40fb-a66b-e771fea8763b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.024643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.025354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.025640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.025920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] PATH_INFO: `/v3/users/a8045692cd1e4d48be5039398bca0cc4` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.027532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] RBAC: Authorizing `identity:get_user(user_id=a8045692cd1e4d48be5039398bca0cc4)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.029109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.030991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 87/172] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/users/a8045692cd1e4d48be5039398bca0cc4 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:41.031606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:41.032669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:41.037506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed9905-9c90-4247-82aa-ca3d31e49cc0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.102803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.103440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.103724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.104002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.142789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d56df-0df5-4abf-b3d1-249b02c1183b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.160028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:41.182078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.183237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.184406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.184987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.187482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.192938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.194842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 88/173] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:41.195975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:41.233429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:41.265753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.267431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.306229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96deb03-f952-481d-8e9e-adf9d1e6882c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.356940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.357911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.358493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.358960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.431339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.433786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.493509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:41.541731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 89/174] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:41.543023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:41.617652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 86/175] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:41.618663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:41.663998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db690eb-d1bf-40da-9137-1d1f774d0618 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.704010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:41.718406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.720192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.720651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.721110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.741983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=40886c602f2b4c218564e5847aed724b) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:41.741983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.741983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.742228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:41.742313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 90/176] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/role_assignments?user.id=40886c602f2b4c218564e5847aed724b&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:41.743369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:41.784811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1395d-9d75-47f4-be5a-9da26532988c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.849921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.850532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.851193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.851193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.857692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.859151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.865296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 87/177] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:41.871715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:41.914728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38baf-21c5-4761-ba39-74299442bd17 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:41.968555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:41.970164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:41.970793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:41.971268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] PATH_INFO: `/v3/projects/a1389069273341eaa0c38004e21d5456` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:41.976721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] RBAC: Authorizing `identity:get_project(project_id=a1389069273341eaa0c38004e21d5456)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:41.978633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:41.980581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 91/178] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/projects/a1389069273341eaa0c38004e21d5456 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:41.986845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:42.004455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f2424-30d7-4e8b-a35c-7ef2d4684d1b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:42.040829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:42.049689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.049994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.050237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.051062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] RBAC: Adding query filter params (role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=a8045692cd1e4d48be5039398bca0cc4) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:42.051307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:42.052791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2cdd1-7382-4391-8855-7622ad2c545f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:42.054800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:42.060862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 88/179] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/role_assignments?user.id=a8045692cd1e4d48be5039398bca0cc4&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:42.094405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:42.116284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017960ea-333f-465d-a5bc-700dffb4978e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:42.155640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:42.156626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.157256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.157742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.158794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=a1389069273341eaa0c38004e21d5456, user.id=a8045692cd1e4d48be5039398bca0cc4) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:42.159358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:42.161163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4ab61-eafa-41e1-8131-28b592665cc7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:42.168681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 92/180] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:41 2022] GET /identity/v3/role_assignments?user.id=a8045692cd1e4d48be5039398bca0cc4&scope.project.id=a1389069273341eaa0c38004e21d5456&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:42.452860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236001be-b083-4757-b07c-f21bffbdc484 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.453226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236001be-b083-4757-b07c-f21bffbdc484 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.453410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236001be-b083-4757-b07c-f21bffbdc484 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.456454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 89/181] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:42.462075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.462544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.462620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.482893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:42.484097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:42.577531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 93/182] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:42.604493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236001be-b083-4757-b07c-f21bffbdc484 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:42.724008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236001be-b083-4757-b07c-f21bffbdc484 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:42.763388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d1420-47cf-429e-9716-889008fbddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:42.764086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d1420-47cf-429e-9716-889008fbddf8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.764374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d1420-47cf-429e-9716-889008fbddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.764647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d1420-47cf-429e-9716-889008fbddf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.775621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d1420-47cf-429e-9716-889008fbddf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-053d1420-47cf-429e-9716-889008fbddf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:42.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:42.782642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:42.783417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 90/183] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:42.790168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:42.858236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:42.896323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33379cca-0417-4bd4-a420-119a2c613b4d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.896629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33379cca-0417-4bd4-a420-119a2c613b4d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.896870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33379cca-0417-4bd4-a420-119a2c613b4d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.898252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 91/184] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:42.903361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c385b5-12b9-4234-a417-3d5fce1ade9e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.903639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c385b5-12b9-4234-a417-3d5fce1ade9e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.903908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c385b5-12b9-4234-a417-3d5fce1ade9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.905434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb0735-9582-468a-b956-9f63297e0725 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:42.922102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-58c385b5-12b9-4234-a417-3d5fce1ade9e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:42.924005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-58c385b5-12b9-4234-a417-3d5fce1ade9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:42.942914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:42.943537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.943821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.944100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.944774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:42.945105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:42.946488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:42.959374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 94/185] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:42.964691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:42.997505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.005547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 92/186] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:43.008124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d46b9-c145-4820-99cb-b6eb687fa2bd None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.008546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d46b9-c145-4820-99cb-b6eb687fa2bd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.008823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d46b9-c145-4820-99cb-b6eb687fa2bd None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.010473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 93/187] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:43.019051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40f248-8325-464a-9424-05203df6e58f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.019459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40f248-8325-464a-9424-05203df6e58f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.019837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40f248-8325-464a-9424-05203df6e58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.039109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ff40f248-8325-464a-9424-05203df6e58f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:43.040397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ff40f248-8325-464a-9424-05203df6e58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:43.082552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc824f4-c8d2-474b-a0a5-3c8f25e5bc6f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.101448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.109295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 94/188] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:43.110902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40f248-8325-464a-9424-05203df6e58f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.125990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.127686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.128195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.129205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] PATH_INFO: `/v3/users/e0b994d3fdf14b548292a0911ebf3c91` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.146893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] RBAC: Authorizing `identity:get_user(user_id=e0b994d3fdf14b548292a0911ebf3c91)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.148420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.150310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 95/189] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:42 2022] GET /identity/v3/users/e0b994d3fdf14b548292a0911ebf3c91 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:43.153857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.214890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40f248-8325-464a-9424-05203df6e58f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.231293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.247071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.247750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.247971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.248246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.257886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901e912-7f48-4af9-a04c-15f3aa326f4a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.259218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:43.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:43.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 95/190] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:43.265974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.292914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fd87b-a6d5-4e37-a6fa-a8989b10f141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.293584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fd87b-a6d5-4e37-a6fa-a8989b10f141 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.293867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fd87b-a6d5-4e37-a6fa-a8989b10f141 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.294142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fd87b-a6d5-4e37-a6fa-a8989b10f141 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.295275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fd87b-a6d5-4e37-a6fa-a8989b10f141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.296703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fd87b-a6d5-4e37-a6fa-a8989b10f141 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.298693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 96/191] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:43.299874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde8aac-827b-4509-bdcf-b32bd8ffa4bd None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.300160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde8aac-827b-4509-bdcf-b32bd8ffa4bd None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.300432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde8aac-827b-4509-bdcf-b32bd8ffa4bd None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.301790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 97/192] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:43.302868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde8aac-827b-4509-bdcf-b32bd8ffa4bd None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.348831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.369167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adc9b1-f025-4987-91d0-9ea0f26bf7ad None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.404464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.405090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.405367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.405633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.411564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde8aac-827b-4509-bdcf-b32bd8ffa4bd None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.445415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.446020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.446298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.446570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.447235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:43.447548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.448907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919bb8c-6a1d-4cc2-9661-19f74d012434 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.452104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.462044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 98/193] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:43.462044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.462044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.462044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.476724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:43.477943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:43.478554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.480017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.544085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 99/194] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:43.544899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.580419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.585313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 96/195] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:43.586744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.644745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37118a6-f784-49c8-bd31-90ca68fa43a5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.668971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.678776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.679413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.679671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.679948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.680583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.682048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.689768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fb94f-756c-4bb0-8c09-4c3b9f83c089 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.724624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.725371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.725578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.725852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] PATH_INFO: `/v3/users/e0b994d3fdf14b548292a0911ebf3c91` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.726984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b994d3fdf14b548292a0911ebf3c91)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.728462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.730047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 97/196] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/users/e0b994d3fdf14b548292a0911ebf3c91 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:43.730801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.764113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 100/197] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:43 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:43.768826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.845101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fe047-1a2c-4c49-8e14-a314d652c7f2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.879224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.884367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.885045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.885329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.885597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.889282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764b784-3820-47ef-918b-2e7357d6d338 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:43.898301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:43.903495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:43.904588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:43.905436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 98/198] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:43.906081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:43.919266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:43.937419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:43.938309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:43.939682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:43.941033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:43.942525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:43.943903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:43.946170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 101/199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:43.948904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.033850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491c36e-592a-4efc-b35f-fa2358221c11 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.078395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecadae9-4945-444b-8b81-711b6a5ffd1a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.084465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.085448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.085909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.086337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.129731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b80d-6549-4267-becd-799a720df43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.130347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b80d-6549-4267-becd-799a720df43b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.130661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b80d-6549-4267-becd-799a720df43b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.130904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b80d-6549-4267-becd-799a720df43b None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.131577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b80d-6549-4267-becd-799a720df43b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:44.131885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b80d-6549-4267-becd-799a720df43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.141483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b80d-6549-4267-becd-799a720df43b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:44.157328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 102/200] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:44.158498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b80d-6549-4267-becd-799a720df43b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.184046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.185591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:44.193815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:44.196870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:44.298965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b80d-6549-4267-becd-799a720df43b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.301172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 99/201] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:44.301828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.350035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.351108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.351568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.352235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.353535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=e0b994d3fdf14b548292a0911ebf3c91) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:44.353851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.355311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:44.363800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 103/202] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:44 2022] GET /identity/v3/role_assignments?user.id=e0b994d3fdf14b548292a0911ebf3c91&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:44.364647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.427548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d490e-186e-4576-85ba-1136c9e93e2e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.436023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:44.470185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.470850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.471110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.471378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] PATH_INFO: `/v3/users/47d9f41513624d38a8773a1b23aeffd7` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.473698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] RBAC: Authorizing `identity:get_user(user_id=47d9f41513624d38a8773a1b23aeffd7)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.475272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:44.477419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 100/203] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:44 2022] GET /identity/v3/users/47d9f41513624d38a8773a1b23aeffd7 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:44.482984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.577849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:44.595456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdad308-02d2-4410-b162-cfd9f1b7bf50 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.631907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2445750-267e-4145-aa2e-f93195f9b1db None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.668790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.669579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.669978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.670343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.673898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.674703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.675118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.681095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.682563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.684177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:44.686025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 104/204] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:44 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:44.694616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.723034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:44.749492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.752475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:44.858015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 101/205] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:44.866089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:44.946188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114678-1151-4ac9-878d-c01f01d128ab None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:44.994948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:44.995572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:44.995855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:44.996128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:44.996862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] RBAC: Adding query filter params (role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=e0b994d3fdf14b548292a0911ebf3c91) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:44.997217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:44.998681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:45.006885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 105/206] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:44 2022] GET /identity/v3/role_assignments?user.id=e0b994d3fdf14b548292a0911ebf3c91&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:45.017442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:45.071990 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de695-5866-4569-b70c-5654768b93f5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:45.124044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6edce2-54d5-4182-8c78-19a742d21a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:45.124698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6edce2-54d5-4182-8c78-19a742d21a85 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:45.125061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6edce2-54d5-4182-8c78-19a742d21a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:45.125442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6edce2-54d5-4182-8c78-19a742d21a85 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:45.126659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6edce2-54d5-4182-8c78-19a742d21a85 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:45.128204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6edce2-54d5-4182-8c78-19a742d21a85 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:45.130285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 102/207] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:44 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:45.136027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:45.165371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:45.277746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bce37-173d-4fd7-8c18-18f75f01208b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:45.314208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:45.314825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:45.315111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:45.315389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:45.316123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] RBAC: Adding query filter params (role.id=3e730d996bc3433bba5409328aabef27, scope.project.id=937d691e04a147e0a4028b930e05179a, user.id=47d9f41513624d38a8773a1b23aeffd7) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:45.316435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:45.317934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e56a-6244-4f7f-8a32-4a0a5d620ca1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:45.324565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 106/208] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:45 2022] GET /identity/v3/role_assignments?user.id=47d9f41513624d38a8773a1b23aeffd7&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=3e730d996bc3433bba5409328aabef27 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:46.096181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbacb4-751a-499f-8172-41db7fb86687 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.096688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbacb4-751a-499f-8172-41db7fb86687 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.096688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbacb4-751a-499f-8172-41db7fb86687 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.098090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 103/209] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:46.103272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077093e0-4164-4b26-b64c-547aa95e5a89 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.103659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077093e0-4164-4b26-b64c-547aa95e5a89 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.103821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077093e0-4164-4b26-b64c-547aa95e5a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.123316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-077093e0-4164-4b26-b64c-547aa95e5a89 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:46.124798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-077093e0-4164-4b26-b64c-547aa95e5a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:46.194926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 107/210] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:46.218340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbacb4-751a-499f-8172-41db7fb86687 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:46.234857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee9632-b0f2-4ed5-85b5-b64981c656a2 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.235235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee9632-b0f2-4ed5-85b5-b64981c656a2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.235422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee9632-b0f2-4ed5-85b5-b64981c656a2 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.236789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 108/211] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:46.241876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.242161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.242438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.261437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:46.262634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:46.270625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:46.332099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbacb4-751a-499f-8172-41db7fb86687 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:46.339887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 109/212] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:46.358986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:46.368573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:46.369509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.369926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.370334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.383658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:46.389285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:46.390295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:46.392024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:46.394435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 104/213] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:46.400191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:46.405408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:46.471626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ab153-6294-4933-8c4d-8b3a0515cce1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:46.512504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:46.513245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.513527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.513845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.514967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:46.517190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:46.518780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 110/214] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:46.524125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:46.537746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:46.561777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f235357-d181-4fb1-bc3b-4a464281549b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:46.611776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:46.613283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.613863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.614346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.615486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:46.616427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:46.618536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:46.630622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 105/215] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:46.636095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:46.640229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8232843-6cf8-4d09-b04b-09fa7e8430b8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:46.682481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:46.683158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.683604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.684018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.684918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:46.686722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:46.723679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:46.762668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a759d-7b6d-4661-a99f-e34ab0244ca7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:46.791913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 111/216] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:46 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:06:46.807102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d1179-5918-4998-9ce2-938a351c2e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:46.808182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1179-5918-4998-9ce2-938a351c2e21 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.808662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1179-5918-4998-9ce2-938a351c2e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.809167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1179-5918-4998-9ce2-938a351c2e21 None admin] PATH_INFO: `/v3/users/69bc6eef6b46424793158dcf1666fe01` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.829300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d1179-5918-4998-9ce2-938a351c2e21 None admin] RBAC: Authorizing `identity:get_user(user_id=69bc6eef6b46424793158dcf1666fe01)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:46.831643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d1179-5918-4998-9ce2-938a351c2e21 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:46.833671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 106/217] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:46 2022] GET /identity/v3/users/69bc6eef6b46424793158dcf1666fe01 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:46.842135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:46.894341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9c03-7c09-46c9-85cc-f532971f9fc7 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.894531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9c03-7c09-46c9-85cc-f532971f9fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.894793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9c03-7c09-46c9-85cc-f532971f9fc7 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.896376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 107/218] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:46.902298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:46.902616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:46.902841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:46.922809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:46.924168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:46.932565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:46.981977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42049865-8278-41fd-b526-dec694f8e4dc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.011666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 108/219] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:47.025147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342a65d-8d19-48be-9490-5fef44d94172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.026157 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a65d-8d19-48be-9490-5fef44d94172 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.026871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a65d-8d19-48be-9490-5fef44d94172 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.027321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a65d-8d19-48be-9490-5fef44d94172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.035058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.105031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:47.110088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342a65d-8d19-48be-9490-5fef44d94172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:47.111631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342a65d-8d19-48be-9490-5fef44d94172 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:47.169577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48591320-a4bb-4114-890c-6ebe5926f214 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.195184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:47.214308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a931-f1ba-4548-a631-ec64b24691fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.214980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a931-f1ba-4548-a631-ec64b24691fe None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.215251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a931-f1ba-4548-a631-ec64b24691fe None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.215838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a931-f1ba-4548-a631-ec64b24691fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.233639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408a931-f1ba-4548-a631-ec64b24691fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:47.239397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 112/220] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:47.240519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5210b-24e1-4d38-9988-e86dd83d7548 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.240804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5210b-24e1-4d38-9988-e86dd83d7548 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.241136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5210b-24e1-4d38-9988-e86dd83d7548 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c408a931-f1ba-4548-a631-ec64b24691fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:47.242123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:47.243404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:47.244364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:47.244957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 113/221] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:47.245684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5210b-24e1-4d38-9988-e86dd83d7548 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.247108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 109/222] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:47.248890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.249413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.250190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.273727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:47.275719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:47.353816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 110/223] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:47.355013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.363747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5210b-24e1-4d38-9988-e86dd83d7548 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.402646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.403374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.403739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.404081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.405519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:47.407360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:47.409734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 114/224] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:47.414685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.416415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:47.459313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:47.470442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e18b85-475d-49ce-8544-6d8017d6a2f0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.510989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.512347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.512972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.513620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.514493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:47.515020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:47.517460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:47.526151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 111/225] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:47.527520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.554425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42578f-19e2-44e1-8ce5-8f92242ea3cc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.601665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.602308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.602745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.603018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.616093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:47.622365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:47.623227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:47.624332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 115/226] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:47.625017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.645499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb9193-6bf8-45e9-9bbe-1f865ba9e1c9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.648502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:47.686400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.687467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.688453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.688907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.690000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=69bc6eef6b46424793158dcf1666fe01) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:47.690512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:47.692176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:47.698678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 112/227] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/role_assignments?user.id=69bc6eef6b46424793158dcf1666fe01&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:47.699666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.801491 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:47.814455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1196d22-8e81-4536-bfb2-566609bd1282 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.887408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.888683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.888963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.889410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] PATH_INFO: `/v3/users/e0b994d3fdf14b548292a0911ebf3c91` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.891574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b994d3fdf14b548292a0911ebf3c91)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:47.892915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:47.894675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 116/228] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/users/e0b994d3fdf14b548292a0911ebf3c91 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:47.901831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:47.913656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63972-26ec-4857-9253-6a379e36c59b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:47.950044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:47.950697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:47.950989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:47.951253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:47.952219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:47.952324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:47.953934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:47.962389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 113/229] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:47.968117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:48.001542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:48.018910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ed483-ea6b-410a-852f-bc159c261579 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:48.055508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:48.056492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:48.056965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:48.057471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:48.058784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:48.060400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:48.062243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 117/230] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:48.068485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:48.074110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a55d5-5b38-4e13-ac77-8da543b97f08 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:48.100979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:48.111407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:48.112042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:48.112321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:48.112643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] PATH_INFO: `/v3/users/e0b994d3fdf14b548292a0911ebf3c91` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:48.113874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] RBAC: Authorizing `identity:get_user(user_id=e0b994d3fdf14b548292a0911ebf3c91)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:48.115330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:48.117020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 114/231] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:47 2022] GET /identity/v3/users/e0b994d3fdf14b548292a0911ebf3c91 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:48.122601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:48.211299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff930cb-6988-4096-ac82-cd744eb848cb None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:48.226694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713a45e-c415-404d-a4c7-1fe6af3fbc9c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:48.229300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:48.263741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:48.264373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:48.264650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:48.264912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:48.266630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:48.268324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:48.270004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 115/232] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:48 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:48.275678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:48.291270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cca3-6a2c-44cb-ad88-ce935dee646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:48.291942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cca3-6a2c-44cb-ad88-ce935dee646b None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:48.292221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cca3-6a2c-44cb-ad88-ce935dee646b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:48.292496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cca3-6a2c-44cb-ad88-ce935dee646b None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/e0b994d3fdf14b548292a0911ebf3c91/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:48.294485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cca3-6a2c-44cb-ad88-ce935dee646b None admin] RBAC: Authorizing `identity:create_grant(project_id=5765aa65f6dd4dac9f062fb2ffa7a762, user_id=e0b994d3fdf14b548292a0911ebf3c91, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:48.296008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cca3-6a2c-44cb-ad88-ce935dee646b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:48.386228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 118/233] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:48 2022] PUT /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/e0b994d3fdf14b548292a0911ebf3c91/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:48.392231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d001c0-52bc-4940-930d-3b0cc4e69eb6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:48.430159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be637f-a12c-4981-9242-d086913cee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:48.430829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be637f-a12c-4981-9242-d086913cee54 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:48.431107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be637f-a12c-4981-9242-d086913cee54 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:48.431380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be637f-a12c-4981-9242-d086913cee54 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/e0b994d3fdf14b548292a0911ebf3c91/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:48.433678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be637f-a12c-4981-9242-d086913cee54 None admin] RBAC: Authorizing `identity:create_grant(project_id=5765aa65f6dd4dac9f062fb2ffa7a762, user_id=e0b994d3fdf14b548292a0911ebf3c91, role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:48.435142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be637f-a12c-4981-9242-d086913cee54 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:48.547864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 116/234] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:48 2022] PUT /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/e0b994d3fdf14b548292a0911ebf3c91/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:49.381545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff254d-481b-45c6-ae89-791e06caec43 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:49.382055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff254d-481b-45c6-ae89-791e06caec43 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:49.382130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff254d-481b-45c6-ae89-791e06caec43 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:49.383451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 119/235] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:49.389287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:49.390015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:49.390015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:49.418015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:49.419238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:49.444751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:49.459058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2131d1-cfe2-42e6-ab61-f1bb88e0dc0f None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:49.459335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2131d1-cfe2-42e6-ab61-f1bb88e0dc0f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:49.459614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2131d1-cfe2-42e6-ab61-f1bb88e0dc0f None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:49.461041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 120/236] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:49.477180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:49.477467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:49.477748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:49.507905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 117/237] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:49.516897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:49.518151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:49.534206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:49.629177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 121/238] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:49.657595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:49.701350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:49.729682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3eec-9180-498b-9461-556e203e5714 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:49.795050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:49.795695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:49.796146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:49.796430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:49.815083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:49.821023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:49.821881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:49.822680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:49.823725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 118/239] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:49.831407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:49.841156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70883184-69b1-43c7-ab0b-3072887c72e9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:49.869555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:49.907547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:49.908271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:49.908494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:49.908768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:49.910152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:49.911685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:49.913421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 122/240] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:49.924652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.027395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72f5fb-767d-4a3c-9831-64e446d57b9e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:50.065391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:50.068389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa9c5e-04d3-41ec-bb5f-a8aea9c26ae7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:50.104593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ca2ba-e372-4486-80d4-2bcfbf39c737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:50.105237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca2ba-e372-4486-80d4-2bcfbf39c737 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.105517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca2ba-e372-4486-80d4-2bcfbf39c737 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.105787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca2ba-e372-4486-80d4-2bcfbf39c737 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.106604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ca2ba-e372-4486-80d4-2bcfbf39c737 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:50.108066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ca2ba-e372-4486-80d4-2bcfbf39c737 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:50.133843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:50.137160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.137160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.137160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.137160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:50.137160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:50.137498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:50.146347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 119/241] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:50.153940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.201987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 123/242] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:49 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:06:50.212261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a99511-05c8-4396-abfd-1c76cccc09ab None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.212715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a99511-05c8-4396-abfd-1c76cccc09ab None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.213672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a99511-05c8-4396-abfd-1c76cccc09ab None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.215455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 124/243] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:50.221921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.222200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.222576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.241969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:50.243289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:50.278787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:50.322390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f7d4f-3ccb-4f3c-aee9-e2426b793bb4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:50.354764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 125/244] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:50.380749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.395735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:50.397299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.397727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.398144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] PATH_INFO: `/v3/users/702e7c56899846508be85e8ba4a8bed8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.430880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] RBAC: Authorizing `identity:get_user(user_id=702e7c56899846508be85e8ba4a8bed8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:50.432382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:50.434373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 120/245] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:50 2022] GET /identity/v3/users/702e7c56899846508be85e8ba4a8bed8 => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:50.442541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.513755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:50.549455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faa9f4-9ac7-426b-8798-a43a20e3dae6 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:50.594991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff630-816d-4d29-9aef-dc12ff97f44a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:50.595496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:50.596218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.596573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.596918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.611951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:50.618207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:50.619097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:50.619893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 126/246] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:50.628786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.659751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129d7c-04bc-400c-9783-379c58584263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:50.660685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129d7c-04bc-400c-9783-379c58584263 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.661221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129d7c-04bc-400c-9783-379c58584263 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.661697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129d7c-04bc-400c-9783-379c58584263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.668632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:50.744882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97129d7c-04bc-400c-9783-379c58584263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:50.748181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97129d7c-04bc-400c-9783-379c58584263 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:50.761574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca030c2-af7c-493c-83aa-db18679aed36 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:50.807155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:50.807809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.808095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.808374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.809353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:50.809662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:50.811235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:50.813528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:50.825085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 127/247] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:50.832553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.894807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 121/248] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:50.897319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd180a0-531a-4e80-a19e-31cbeb52d00d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:50.897319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd180a0-531a-4e80-a19e-31cbeb52d00d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:50.897319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd180a0-531a-4e80-a19e-31cbeb52d00d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:50.898427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 122/249] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:50.901755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd180a0-531a-4e80-a19e-31cbeb52d00d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:50.949777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:50.964760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71c0cb-4db7-42b3-897d-9864549ebfd2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.003688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d709a-0fb7-4d2d-9507-81172936c0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.004328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d709a-0fb7-4d2d-9507-81172936c0ae None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.004605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d709a-0fb7-4d2d-9507-81172936c0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.004878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d709a-0fb7-4d2d-9507-81172936c0ae None admin] PATH_INFO: `/v3/users/69bc6eef6b46424793158dcf1666fe01` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.006037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d709a-0fb7-4d2d-9507-81172936c0ae None admin] RBAC: Authorizing `identity:get_user(user_id=69bc6eef6b46424793158dcf1666fe01)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:51.007528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d709a-0fb7-4d2d-9507-81172936c0ae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:51.009344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 128/250] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:50 2022] GET /identity/v3/users/69bc6eef6b46424793158dcf1666fe01 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:51.010243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.010525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.011131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.030857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd180a0-531a-4e80-a19e-31cbeb52d00d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.033263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:51.035281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:51.076497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037444a-333b-4b98-84dd-5e5ff3653fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.077087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037444a-333b-4b98-84dd-5e5ff3653fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.077367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037444a-333b-4b98-84dd-5e5ff3653fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.077640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037444a-333b-4b98-84dd-5e5ff3653fb4 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.080618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037444a-333b-4b98-84dd-5e5ff3653fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:51.082116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037444a-333b-4b98-84dd-5e5ff3653fb4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:51.083986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 123/251] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:50 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:51.085483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d40fe-f39d-4e8c-bd55-bcdeb3706f94 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.085761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d40fe-f39d-4e8c-bd55-bcdeb3706f94 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.086029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d40fe-f39d-4e8c-bd55-bcdeb3706f94 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.087781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 124/252] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:51.088469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d40fe-f39d-4e8c-bd55-bcdeb3706f94 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.109209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:51.118078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 129/253] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:51.118987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.236893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f66f57-bb35-4d80-be8d-8a08cf982c35 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.246194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d40fe-f39d-4e8c-bd55-bcdeb3706f94 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.249759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:51.273414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.274121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.274471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.274812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.275613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] RBAC: Adding query filter params (role.id=3e730d996bc3433bba5409328aabef27, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=702e7c56899846508be85e8ba4a8bed8) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:51.276048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:51.277868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a15dc-df66-4c14-b7dc-3d0270cbadcf None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:51.285274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 130/254] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/role_assignments?user.id=702e7c56899846508be85e8ba4a8bed8&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=3e730d996bc3433bba5409328aabef27 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:51.286679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.287072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.287439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.290932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.291582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.291858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.292127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.293336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:51.294822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:51.296782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 125/255] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:51.297988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.337791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:51.338959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:51.426583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d284a7-df43-48af-ae83-535c4410b7de None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.435570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:51.461204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 131/256] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:51.462009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.482576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.483560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.484039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.484543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.502691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:51.512211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:51.513169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:06:51.513944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:51.516394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 126/257] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:51.518737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.557384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:51.644008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b6f87-fc5e-46dc-a48a-869dea513060 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.649787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3527f-2f4a-458a-8194-4580f93907d9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.682855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.683834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.684331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.684961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.705328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:51.712616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:51.713560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:06:51.714332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:51.717328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 127/258] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:51.726121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.743094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.744075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.744556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.745075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/69bc6eef6b46424793158dcf1666fe01/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] RBAC: Authorizing `identity:create_grant(project_id=5765aa65f6dd4dac9f062fb2ffa7a762, user_id=69bc6eef6b46424793158dcf1666fe01, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:51.751698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:51.839000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 132/259] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:51 2022] PUT /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/69bc6eef6b46424793158dcf1666fe01/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:51.840826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:51.844789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:51.860420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd4f41-cdd7-471b-809c-23d07bf3dd54 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:51.923848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:51.957396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:51.958033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:51.958320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:51.958594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:51.959374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:51.959683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:51.961112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:51.984547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 128/260] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:51.990322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:52.006586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f09e7e-c3ea-4716-8163-80155c61b050 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:52.074628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:52.076342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:52.076908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:52.077450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:52.079252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:52.079756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:52.081416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:52.097495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 133/261] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:52.103610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:52.247312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b7c64-80e1-403e-a615-77e35c6a9c86 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:52.255428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37962ce-ac02-4251-bf8b-231ab9aa701a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:52.272099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:52.299681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:52.301257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:52.301686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:52.302089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] PATH_INFO: `/v3/users/e0b994d3fdf14b548292a0911ebf3c91` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:52.303665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b994d3fdf14b548292a0911ebf3c91)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:52.305143 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:52.306958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:52.307390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 129/262] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:51 2022] GET /identity/v3/users/e0b994d3fdf14b548292a0911ebf3c91 => generated 283 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:52.308021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:52.308764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:52.309209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] PATH_INFO: `/v3/users/e0b994d3fdf14b548292a0911ebf3c91` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:52.315533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:52.316401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] RBAC: Authorizing `identity:get_user(user_id=e0b994d3fdf14b548292a0911ebf3c91)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:52.317945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:52.319615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 134/263] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:52 2022] GET /identity/v3/users/e0b994d3fdf14b548292a0911ebf3c91 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:52.329883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:52.368401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:52.577231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52500686-d365-48e4-9208-acdf1bac053f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:52.582047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc32ba-923b-4a06-80fb-a3a3fb68b571 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:52.642795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:52.643481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:52.643750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:52.644023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:52.663060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:52.663687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:52.663968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:52.664240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:52.743320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:52.788460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:52.790017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:52.818081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:52.819577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:52.862669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:53.008215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 130/264] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:53.013245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.024859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 135/265] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:53.031016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.167501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d483c-817d-4534-bdc4-c099f89dc9c7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:53.191225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:53.198608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:53.203350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:53.203992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.211279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.211279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.211279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:53.211279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:53.211279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 131/266] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:53.214812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.237039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97412a81-f007-44ef-a1ae-6bcb8e1a78e7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:53.315594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdb37b-811f-4841-9c89-a616639811c5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:53.322393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:53.323053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.323334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.323610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.329716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:53.331174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:53.332843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 136/267] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:53.342226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.355092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:53.355699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.355975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.356247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.356968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] RBAC: Adding query filter params (role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=e0b994d3fdf14b548292a0911ebf3c91) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:53.357304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:53.358724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea331ac-3dc1-4ecf-914d-12f1d1f903ea None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:53.365502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 132/268] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/role_assignments?user.id=e0b994d3fdf14b548292a0911ebf3c91&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:53.373204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935d374-afec-4a1e-94d7-4de82fe2ac94 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.373511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935d374-afec-4a1e-94d7-4de82fe2ac94 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.373753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935d374-afec-4a1e-94d7-4de82fe2ac94 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.375123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 133/269] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:53.380590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae78b9-7257-465a-beb5-fe0f41e35b50 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.380876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae78b9-7257-465a-beb5-fe0f41e35b50 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.381186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae78b9-7257-465a-beb5-fe0f41e35b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.424680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7fae78b9-7257-465a-beb5-fe0f41e35b50 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:53.426605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7fae78b9-7257-465a-beb5-fe0f41e35b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:53.530466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:53.546433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4515b-5d70-4bde-b29c-67eebfcf09e7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:53.566618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 134/270] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:53.596951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:53.597978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.598459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.598924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.599866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=e0b994d3fdf14b548292a0911ebf3c91) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:53.600368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:53.602076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a736a0-6da8-464f-80df-a9997a3fd779 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:53.608801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae350cec-c284-45e8-8a0f-d1c2ca5fdbdc None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.610563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae350cec-c284-45e8-8a0f-d1c2ca5fdbdc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.611292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 137/271] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/role_assignments?user.id=e0b994d3fdf14b548292a0911ebf3c91&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:53.611402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae350cec-c284-45e8-8a0f-d1c2ca5fdbdc None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.613202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 135/272] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:53.626899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.627191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.627465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.652482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:53.652962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae350cec-c284-45e8-8a0f-d1c2ca5fdbdc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.654421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:53.657134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:53.785615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 138/273] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:53.827069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.852596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae350cec-c284-45e8-8a0f-d1c2ca5fdbdc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:53.889539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:53.890171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:53.890453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:53.890756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:53.905365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:53.911743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:53.912642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:53.913456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 136/274] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:53.921617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:53.956662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8f64d-9c87-4402-90c5-a49440b712a1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:53.977346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:54.002970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.017120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.017120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.017120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.017120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:54.017120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:54.017120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 139/275] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:54.021668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.035886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:54.056924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef3711-9934-41d1-a538-50d43783b18f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.103623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.104506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.105215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.106068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.107237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:54.107739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:54.110825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:54.122469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 137/276] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:54.129987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.159820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d76c8-8fad-4f41-b8d3-3f2deba05879 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.196853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61310e95-2c24-4b7a-a0f2-43d0a57ddbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.198265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61310e95-2c24-4b7a-a0f2-43d0a57ddbf1 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.199090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61310e95-2c24-4b7a-a0f2-43d0a57ddbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.199894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61310e95-2c24-4b7a-a0f2-43d0a57ddbf1 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.201393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61310e95-2c24-4b7a-a0f2-43d0a57ddbf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:54.204875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61310e95-2c24-4b7a-a0f2-43d0a57ddbf1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:54.240590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5bc59-0c0d-4d37-9362-d5cc1c201812 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.248306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:54.275428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d51efc-9be3-45c1-a1ea-081be2ce91ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.276117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d51efc-9be3-45c1-a1ea-081be2ce91ab None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.276639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d51efc-9be3-45c1-a1ea-081be2ce91ab None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.276925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d51efc-9be3-45c1-a1ea-081be2ce91ab None admin] PATH_INFO: `/v3/users/5fda3836618d43f49dbcf6e9865bf000` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.290963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d51efc-9be3-45c1-a1ea-081be2ce91ab None admin] RBAC: Authorizing `identity:get_user(user_id=5fda3836618d43f49dbcf6e9865bf000)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:54.292351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d51efc-9be3-45c1-a1ea-081be2ce91ab None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:54.294028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 138/277] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:54 2022] GET /identity/v3/users/5fda3836618d43f49dbcf6e9865bf000 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:54.295179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c538-f57e-43ae-a965-f21aa0c22948 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.295454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c538-f57e-43ae-a965-f21aa0c22948 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.295721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c538-f57e-43ae-a965-f21aa0c22948 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.297125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 139/278] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:54.299946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c538-f57e-43ae-a965-f21aa0c22948 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.311477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 140/279] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:06:54 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:06:54.313276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.316009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.316009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.334460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:54.335849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:54.410239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:54.431501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 141/280] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:54.451382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.463787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c538-f57e-43ae-a965-f21aa0c22948 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.505752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.506712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.507185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.507634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.508634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, user.id=5fda3836618d43f49dbcf6e9865bf000, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:54.509170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:54.510972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:54.516959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 140/281] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 10 13:06:54 2022] GET /identity/v3/role_assignments?user.id=5fda3836618d43f49dbcf6e9865bf000&scope.system=all&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:54.588848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0626f8-0daa-4b97-a451-86b10ce2abb1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.630112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a991d-0529-4086-9fdc-f8e26741587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.630777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a991d-0529-4086-9fdc-f8e26741587e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.631161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a991d-0529-4086-9fdc-f8e26741587e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.631439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a991d-0529-4086-9fdc-f8e26741587e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.643836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a991d-0529-4086-9fdc-f8e26741587e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-884a991d-0529-4086-9fdc-f8e26741587e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:54.648885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:54.649865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:54.650660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 142/282] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:06:54.659054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.665639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90816a2-f487-4a7c-aec1-5721ea1c0e66 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.666200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90816a2-f487-4a7c-aec1-5721ea1c0e66 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.666652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90816a2-f487-4a7c-aec1-5721ea1c0e66 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.668516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 143/283] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:54.674570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.675060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.675530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.696779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:54.698238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:54.704408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:54.760099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:54.786551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 144/284] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:54.810322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.816700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29813d9-3883-4d6f-bb0e-601373250daa None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.853324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:54.854040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:54.855194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:54.855541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:54.856278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:54.856655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:54.858122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:54.868893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 141/285] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:54.875383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:54.955192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115bf47-629d-4065-b567-5bb3ff8f6fca None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:54.991312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:55.002611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f6ec4-42f8-49f2-b9c7-3c5f5b1c82b8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:55.043839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.044487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.044776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.046086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:55.058132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.058777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.059058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.059333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] PATH_INFO: `/v3/users/702e7c56899846508be85e8ba4a8bed8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:55.061182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] RBAC: Authorizing `identity:get_user(user_id=702e7c56899846508be85e8ba4a8bed8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:55.064396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:55.066286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 142/286] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:54 2022] GET /identity/v3/users/702e7c56899846508be85e8ba4a8bed8 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:55.067546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:55.072463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:55.073377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:55.074145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 145/287] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:55.074797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:55.090659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:55.182698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:55.244729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4f14-da63-4933-aba2-22f47f1534d2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:55.272544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.273206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.273561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.273762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:55.274899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:55.283503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:55.283503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 143/288] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:55 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:55.284514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:55.293492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7af22-8af3-4e80-8de4-d3c63e500c7b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:55.353462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:55.378229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.379130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.379556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.379960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:55.380761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:55.381228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:55.382747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:55.401027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625791b-c9d6-4c24-bc7f-1b4bcb343689 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:55.403517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 146/289] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:55.415563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:55.442767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.443528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.443876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.444253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/702e7c56899846508be85e8ba4a8bed8/roles/3e730d996bc3433bba5409328aabef27` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:55.455719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] RBAC: Authorizing `identity:create_grant(project_id=5765aa65f6dd4dac9f062fb2ffa7a762, user_id=702e7c56899846508be85e8ba4a8bed8, role_id=3e730d996bc3433bba5409328aabef27)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:55.458276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:55.535289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 144/290] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:06:55 2022] PUT /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/users/702e7c56899846508be85e8ba4a8bed8/roles/3e730d996bc3433bba5409328aabef27 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:06:55.596321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e621fae-c4eb-4db1-b0c2-901c2868997e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:55.607610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:55.653418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.654083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.654366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.654645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] PATH_INFO: `/v3/users/69bc6eef6b46424793158dcf1666fe01` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:55.655853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] RBAC: Authorizing `identity:get_user(user_id=69bc6eef6b46424793158dcf1666fe01)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:55.657932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:55.659793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 147/291] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:55 2022] GET /identity/v3/users/69bc6eef6b46424793158dcf1666fe01 => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:55.666604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:55.808305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:55.834460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e71c39-e7d1-4f83-9972-b80c0566f62b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:55.886036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:55.886996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:55.887466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:55.887773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:56.013749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:56.015239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:56.121129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:56.151293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 145/292] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:56.158503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:56.297746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3126fd-8a6d-4391-807a-b54642161be5 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:56.342654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd50a44-80ff-4396-8476-a851c90faf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:56.343315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd50a44-80ff-4396-8476-a851c90faf15 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:56.343602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd50a44-80ff-4396-8476-a851c90faf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:56.344431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd50a44-80ff-4396-8476-a851c90faf15 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:56.345724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd50a44-80ff-4396-8476-a851c90faf15 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:56.347193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd50a44-80ff-4396-8476-a851c90faf15 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:56.348869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 148/293] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:56 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:56.355426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:56.501331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fe74a-ce4e-4419-b262-bda514e3c16b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:56.553736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:56.554365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:56.554648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:56.554946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:56.555988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=69bc6eef6b46424793158dcf1666fe01) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:56.556524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:56.558491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b8c92-0a80-4990-999a-8fa32d8052bd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:56.565972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 146/294] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:56 2022] GET /identity/v3/role_assignments?user.id=69bc6eef6b46424793158dcf1666fe01&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:56.949823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40be685-e220-48d3-ae8e-feb67db1562f None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:56.950213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40be685-e220-48d3-ae8e-feb67db1562f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:56.950389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40be685-e220-48d3-ae8e-feb67db1562f None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:56.952004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 149/295] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:56.958118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:56.958578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:56.958652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:56.979391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:56.980560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:57.035025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.048646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b017-620c-49b0-ac45-ca27c790bb93 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.048917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b017-620c-49b0-ac45-ca27c790bb93 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.049229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b017-620c-49b0-ac45-ca27c790bb93 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.050573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 150/296] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:57.056273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.057875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.057875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.063569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 147/297] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:57.077575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:57.079391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:57.095833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.125530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.169665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 151/298] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:06:57.194240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.216661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a494908-9e07-46c8-a251-0d3d4a4f3f76 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.251721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.252412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.252696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.252970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.263004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:57.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:57.269319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:57.270115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 148/299] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:57.277120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.313436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14815a47-7b5a-4cef-aade-12d090c44857 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.319575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.354821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.355810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.356332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.356811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.368581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:57.374105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:57.374967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:06:57.376274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:06:57.378853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 152/300] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:57.383854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac333ee2-e559-4b3f-97f7-a15feb3a8a39 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.385976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.415073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.415686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.415971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.416246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.416970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:57.417314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:57.418687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:57.424621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.431103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 149/301] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:57.433417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.511779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ed92-6cbb-4c48-80d9-ccbb3262b3fa None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.550329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.550938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.551217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.551487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.551932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57248-458c-42f4-b1aa-eb3ebed8e647 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.552152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:57.552453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:57.553462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.553858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e570a-6bef-4bbb-b7bb-f8ce0d3e336a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:57.562167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 153/302] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:57.563409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891dc11-e72c-41db-a4e6-2cc905e4867d None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.563693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891dc11-e72c-41db-a4e6-2cc905e4867d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.563967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891dc11-e72c-41db-a4e6-2cc905e4867d None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.565408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 154/303] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:06:57.569866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891dc11-e72c-41db-a4e6-2cc905e4867d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.598283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6bf0a-8e09-40e5-af26-489b73df7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.599034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6bf0a-8e09-40e5-af26-489b73df7763 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.599314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6bf0a-8e09-40e5-af26-489b73df7763 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.599591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6bf0a-8e09-40e5-af26-489b73df7763 None admin] PATH_INFO: `/v3/users/79908b068149418e8483356ffbfe71e2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.614317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6bf0a-8e09-40e5-af26-489b73df7763 None admin] RBAC: Authorizing `identity:get_user(user_id=79908b068149418e8483356ffbfe71e2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:57.615980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6bf0a-8e09-40e5-af26-489b73df7763 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:57.617781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 150/304] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/users/79908b068149418e8483356ffbfe71e2 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:06:57.619041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.619443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.619836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.637826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:57.639220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:57.674511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.682024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891dc11-e72c-41db-a4e6-2cc905e4867d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.716489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.717168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.717453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.717729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] PATH_INFO: `/v3/users/5fda3836618d43f49dbcf6e9865bf000` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.719367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] RBAC: Authorizing `identity:get_user(user_id=5fda3836618d43f49dbcf6e9865bf000)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:57.720027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 151/305] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:57.720814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:57.727120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 155/306] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/users/5fda3836618d43f49dbcf6e9865bf000 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:57.727247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.734812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.822066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:57.831363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35538a-3de3-4890-8f48-6250706f1f42 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.849208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8887c-880c-4ba1-9c42-1bc7e4a036f2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:57.866870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.867481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.867755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.868025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.868746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] RBAC: Adding query filter params (role.id=3e730d996bc3433bba5409328aabef27, user.id=79908b068149418e8483356ffbfe71e2, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:57.869086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:57.870513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:57.875931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 152/307] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/role_assignments?user.id=79908b068149418e8483356ffbfe71e2&scope.system=all&role.id=3e730d996bc3433bba5409328aabef27 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:57.876539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:57.889776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:57.891883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:57.892401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:57.893233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] PATH_INFO: `/v3/system/users/5fda3836618d43f49dbcf6e9865bf000/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:57.895981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5fda3836618d43f49dbcf6e9865bf000, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:57.898486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:57.990867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cdd51-9b82-4ff6-9743-21951b09240d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:58.001390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 156/308] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Aug 10 13:06:57 2022] PUT /identity/v3/system/users/5fda3836618d43f49dbcf6e9865bf000/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:06:58.035962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fe10d-fcaf-48bc-b5b1-94488f21824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:58.036613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe10d-fcaf-48bc-b5b1-94488f21824a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.036898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe10d-fcaf-48bc-b5b1-94488f21824a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.037201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe10d-fcaf-48bc-b5b1-94488f21824a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.038353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fe10d-fcaf-48bc-b5b1-94488f21824a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:58.039836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fe10d-fcaf-48bc-b5b1-94488f21824a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:58.041845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 153/309] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:06:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:58.048534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:58.060244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c01fc-fe61-4426-b83b-3f47a4912e7b None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.060562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c01fc-fe61-4426-b83b-3f47a4912e7b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.060800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c01fc-fe61-4426-b83b-3f47a4912e7b None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.062230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 154/310] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:06:58.068873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.069197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.069471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.082578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:58.089613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:06:58.091159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:58.114411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:58.181160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 155/311] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:06:58.182214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f1dc8-6ab9-4769-b1bb-7cf499d6f451 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:58.198968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:58.237302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:58.238284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.239018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.240409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] PATH_INFO: `/v3/groups` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.241336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:58.242991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:58.309160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dac1b7-d481-499a-a881-ece95b5af254 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:58.324734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 157/312] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Aug 10 13:06:58 2022] POST /identity/v3/groups => generated 229 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:06:58.359674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:58.360655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.361051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.361329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.373820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:06:58.379951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:06:58.380887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:06:58.381940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 156/313] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:06:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:06:58.389364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:58.500918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:58.515755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25615-38bb-49ed-83d2-ced9cb2e5eae None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:58.550740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:58.551351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.551629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.551899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.552567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:58.552868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:58.554428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:58.564053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 158/314] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:06:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:58.569767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:58.589926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:58.679956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54634884-1d4b-40c8-bd30-26b82da570c1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:58.714819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:58.715449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.715724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.715991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] PATH_INFO: `/v3/users/702e7c56899846508be85e8ba4a8bed8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.717170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] RBAC: Authorizing `identity:get_user(user_id=702e7c56899846508be85e8ba4a8bed8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:58.719069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:58.720950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 157/315] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:06:58 2022] GET /identity/v3/users/702e7c56899846508be85e8ba4a8bed8 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:58.726197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:58.830336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126acea-9e47-400c-b4af-6ff57f35687c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:58.866180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:58.866795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:58.867077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:58.867348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:58.885373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:58.935688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:58.937113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:59.044008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 159/316] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:06:59.045417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:59.134114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:59.158857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9d04d-82cb-44b3-a3c1-e37fd2ce1559 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:59.194419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b1a09-691f-4c8c-a1b7-3f8c1dec1db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:59.195943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b1a09-691f-4c8c-a1b7-3f8c1dec1db5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:59.195943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b1a09-691f-4c8c-a1b7-3f8c1dec1db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:59.195943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b1a09-691f-4c8c-a1b7-3f8c1dec1db5 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:59.196974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b1a09-691f-4c8c-a1b7-3f8c1dec1db5 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:59.198648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b1a09-691f-4c8c-a1b7-3f8c1dec1db5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:59.200796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 158/317] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:06:59 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:06:59.209029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:59.269556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:59.315376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36150-779b-4bd0-b03b-e0d96d407e5d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:59.353126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:59.353731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:59.354013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:59.354382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:59.355114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] RBAC: Adding query filter params (role.id=3e730d996bc3433bba5409328aabef27, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762, user.id=702e7c56899846508be85e8ba4a8bed8) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:06:59.355424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:59.356848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902b378-d96c-4f6f-b78a-fa0eb235eaea None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:59.363138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 160/318] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:06:59 2022] GET /identity/v3/role_assignments?user.id=702e7c56899846508be85e8ba4a8bed8&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=3e730d996bc3433bba5409328aabef27 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:00.035053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c16d-5f5d-4207-b1fc-e68baf86587b None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.035313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c16d-5f5d-4207-b1fc-e68baf86587b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.035604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c16d-5f5d-4207-b1fc-e68baf86587b None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.037110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 159/319] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:00.041905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4b33-7ed1-4117-a0d5-eec64c6628a4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.042200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4b33-7ed1-4117-a0d5-eec64c6628a4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.042472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4b33-7ed1-4117-a0d5-eec64c6628a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.061167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5f3c4b33-7ed1-4117-a0d5-eec64c6628a4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:00.062358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5f3c4b33-7ed1-4117-a0d5-eec64c6628a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:00.130841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 161/320] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:00.131967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4685537-6636-4962-91ef-92b07de3fac8 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.132399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4685537-6636-4962-91ef-92b07de3fac8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.132507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4685537-6636-4962-91ef-92b07de3fac8 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.135775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 160/321] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:00.138751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.139042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.139315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.152751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4685537-6636-4962-91ef-92b07de3fac8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.159465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:00.160650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:00.199969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.235925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 162/322] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:00.260932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4685537-6636-4962-91ef-92b07de3fac8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.261942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.298045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.298652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.298932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.299202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.309296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:00.314769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:00.315634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:00.316400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 161/323] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:00.321526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.333204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.371313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7892b2f-3310-4696-a6da-3f127d739e93 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.409186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.409825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.410101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.410373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.420913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:00.426178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:00.427230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:00.428321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:00.430913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a17916-2335-405d-912e-b20af7af47b6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.432437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 163/324] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:00.433406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.466809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.468149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.469405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.469685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.469952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.470621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:00.470921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:00.472406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3907cb-eb65-4221-9839-6c9a62a2d396 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:00.480444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 162/325] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:00.480444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8dbc6-881f-45ff-953b-ca765952092d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.480444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8dbc6-881f-45ff-953b-ca765952092d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.480444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8dbc6-881f-45ff-953b-ca765952092d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.480444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 163/326] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:00.486936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8dbc6-881f-45ff-953b-ca765952092d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.552318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737077a-fb66-4ad7-8ada-54bf254c1044 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.558651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.587697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.588294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.588569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.588840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.589530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:00.589828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:00.591218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db18b4a-bbba-4c9a-a8be-5150a011bbfa None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:00.591588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8dbc6-881f-45ff-953b-ca765952092d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.600119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 164/327] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:00.601297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.601579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.601849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.619143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:00.620309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:00.626676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.627303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.627586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.627860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] PATH_INFO: `/v3/users/79908b068149418e8483356ffbfe71e2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.628978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] RBAC: Authorizing `identity:get_user(user_id=79908b068149418e8483356ffbfe71e2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:00.630433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:00.632012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 164/328] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/users/79908b068149418e8483356ffbfe71e2 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:00.632763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.691419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 165/329] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:00.697221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.720836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.740554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c2716-b738-4826-96a9-fc6001cfa33d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.773733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.774350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.774628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.774900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] PATH_INFO: `/v3/users/5fda3836618d43f49dbcf6e9865bf000` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.776003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] RBAC: Authorizing `identity:get_user(user_id=5fda3836618d43f49dbcf6e9865bf000)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:00.777715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:00.779281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 165/330] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/users/5fda3836618d43f49dbcf6e9865bf000 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:00.785499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.793710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e3f56-c3c2-4528-8a04-1b2ae867c372 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.793846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.828684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.829271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.829553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.829826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] PATH_INFO: `/v3/system/users/79908b068149418e8483356ffbfe71e2/roles/3e730d996bc3433bba5409328aabef27` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.831226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=79908b068149418e8483356ffbfe71e2, role_id=3e730d996bc3433bba5409328aabef27)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:00.832591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:00.873022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 166/331] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Aug 10 13:07:00 2022] PUT /identity/v3/system/users/79908b068149418e8483356ffbfe71e2/roles/3e730d996bc3433bba5409328aabef27 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:00.874134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.885437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a5aa6-b225-475a-9c1e-21ae80966c5d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:00.929063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:00.929732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:00.930026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:00.930290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:00.931492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:00.932967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:00.934829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 166/332] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:00.942494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:00.983662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:00.998994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302af85-2449-4629-af7d-5227aa563fbc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:01.025879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:01.041010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:01.041636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:01.041906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:01.042176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:01.043305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] RBAC: Adding query filter params (role.id=31c6a032fe034fd5abc43cc4ce065209, user.id=5fda3836618d43f49dbcf6e9865bf000, scope.system=all) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:01.043305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:01.044715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ba17a-7933-4580-bacc-a564ad08110d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:01.055444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5309c-abad-49f5-8122-3f06d05c9d1c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:01.057464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 167/333] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 10 13:07:00 2022] GET /identity/v3/role_assignments?user.id=5fda3836618d43f49dbcf6e9865bf000&scope.system=all&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:01.098150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2b465-b7f9-4858-916c-24d5d032ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:01.098692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2b465-b7f9-4858-916c-24d5d032ab57 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:01.098975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2b465-b7f9-4858-916c-24d5d032ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:01.099249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2b465-b7f9-4858-916c-24d5d032ab57 None admin] PATH_INFO: `/v3/groups` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:01.099891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2b465-b7f9-4858-916c-24d5d032ab57 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:01.101378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2b465-b7f9-4858-916c-24d5d032ab57 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:01.177147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 167/334] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Aug 10 13:07:00 2022] POST /identity/v3/groups => generated 226 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:07:02.952436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b752-a312-47aa-8bdd-521fa16cd2d9 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:02.952923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b752-a312-47aa-8bdd-521fa16cd2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:02.952923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b752-a312-47aa-8bdd-521fa16cd2d9 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:02.954645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 168/335] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:02.960578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:02.961032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:02.961162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:02.981101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:02.982310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:03.052547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 168/336] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:03.087521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b752-a312-47aa-8bdd-521fa16cd2d9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:03.209773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b752-a312-47aa-8bdd-521fa16cd2d9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:03.251009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:03.251826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:03.252105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:03.252380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:03.265323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:03.271284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:03.273745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:03.275636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 169/337] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:03.282542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:03.370985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:03.413232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed15805-8e22-482b-81ca-50ddd1dbf12a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:03.452586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:03.453785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:03.454102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:03.454323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:03.455264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:03.455264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:03.456657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:03.468388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 169/338] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:03.474642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:03.485234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:03.600265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdebc6-c9be-44f6-91fc-dded83c0c121 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:03.641217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07273f-754c-41c1-a9ea-6364caecd2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:03.643309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07273f-754c-41c1-a9ea-6364caecd2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:03.643598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07273f-754c-41c1-a9ea-6364caecd2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:03.643873 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07273f-754c-41c1-a9ea-6364caecd2c3 None admin] PATH_INFO: `/v3/users/79908b068149418e8483356ffbfe71e2` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:03.655423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07273f-754c-41c1-a9ea-6364caecd2c3 None admin] RBAC: Authorizing `identity:get_user(user_id=79908b068149418e8483356ffbfe71e2)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:03.655423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07273f-754c-41c1-a9ea-6364caecd2c3 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:03.655423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 170/339] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:07:03 2022] GET /identity/v3/users/79908b068149418e8483356ffbfe71e2 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:03.664551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:03.784297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6816bf3-1532-4682-b944-9104ee0ba7ca None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:03.810944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:03.819974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:03.820485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:03.820756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:03.821977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:03.821977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] RBAC: Adding query filter params (role.id=3e730d996bc3433bba5409328aabef27, user.id=79908b068149418e8483356ffbfe71e2, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:03.822289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:03.823526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13a884-3500-4c5e-9172-dada14a4756a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:03.839076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 170/340] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 10 13:07:03 2022] GET /identity/v3/role_assignments?user.id=79908b068149418e8483356ffbfe71e2&scope.system=all&role.id=3e730d996bc3433bba5409328aabef27 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:03.936504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e73d62-ef3c-4fc3-81e5-9e1f061ee093 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:03.936852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e73d62-ef3c-4fc3-81e5-9e1f061ee093 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:03.937058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e73d62-ef3c-4fc3-81e5-9e1f061ee093 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:03.938834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 171/341] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:03.945271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333ad7b-6434-451b-9ab9-e3928ba2e3d2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:03.945580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333ad7b-6434-451b-9ab9-e3928ba2e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:03.945818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333ad7b-6434-451b-9ab9-e3928ba2e3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:03.974158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a333ad7b-6434-451b-9ab9-e3928ba2e3d2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:03.975487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a333ad7b-6434-451b-9ab9-e3928ba2e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:04.053895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 171/342] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:04.092569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e73d62-ef3c-4fc3-81e5-9e1f061ee093 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:04.193044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:04.224533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e73d62-ef3c-4fc3-81e5-9e1f061ee093 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:04.238488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9666da0-334f-4906-9e40-b0fe5bab7d96 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.238778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9666da0-334f-4906-9e40-b0fe5bab7d96 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.239030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9666da0-334f-4906-9e40-b0fe5bab7d96 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.240613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 172/343] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:04.246440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a409d7-624b-449a-84f5-0f33848e1b2f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.246723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a409d7-624b-449a-84f5-0f33848e1b2f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.247001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a409d7-624b-449a-84f5-0f33848e1b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.271083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:04.271787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.272066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.272390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.290124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d5a409d7-624b-449a-84f5-0f33848e1b2f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:04.290956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:04.291355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d5a409d7-624b-449a-84f5-0f33848e1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:04.296588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:04.298133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:04.298913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:04.301683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 172/344] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:04.307784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:04.384329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 173/345] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:04.385452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b2c0-6629-48db-9712-982eaeabcdc6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.385737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b2c0-6629-48db-9712-982eaeabcdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.386008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b2c0-6629-48db-9712-982eaeabcdc6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.387254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 174/346] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:04.395621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd33a-4695-466e-90dc-474843e931be None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.395906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd33a-4695-466e-90dc-474843e931be None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.396180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd33a-4695-466e-90dc-474843e931be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.421185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-556bd33a-4695-466e-90dc-474843e931be None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:04.422694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-556bd33a-4695-466e-90dc-474843e931be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:04.441907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206da5e-4242-4158-b413-136d162f1b0a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:04.466957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:04.482031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:04.482884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.483262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.483634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.484697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:04.485352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:04.486996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:04.488455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:04.501185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 175/348] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:04.501403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 173/348] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:04.502381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:04.508129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bd33a-4695-466e-90dc-474843e931be None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:04.635260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bd33a-4695-466e-90dc-474843e931be None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:04.636068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bbfcb-ee5f-4164-a845-ada8cb16947e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:04.712707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:04.713399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.713678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.713949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.723018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:04.723018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.723018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.723402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:04.729901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:04.735564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:04.736625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:04.737514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:04.739454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 174/349] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:04.740411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:04.799199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:04.830888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:04.833375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:04.835043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:04.915713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74978c-afc6-44f5-b0c9-c22f079e36b9 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:04.986996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:04.987762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:04.988116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:04.988460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.005547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:05.011572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:05.012453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:05.013370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 175/350] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:05.014057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.055477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 176/351] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:05.062037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.145581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037520cc-1515-4016-9670-24ab809d41d9 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.167178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:05.191846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.193351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:05.194327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.194769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.195196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.197818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:05.197818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:05.198794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:05.210267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 176/352] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:05.211368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.239300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d007e-9392-438f-b93f-845210ab4ce1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.296541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.297824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.298188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.298316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.298986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:05.299288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:05.300661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dafbdb-44bf-4d57-a41c-81c5046dfcb6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:05.315437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 177/353] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:05.315437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c456-eacb-4d4e-8fe8-50300db5d1c4 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.315437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c456-eacb-4d4e-8fe8-50300db5d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.315437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c456-eacb-4d4e-8fe8-50300db5d1c4 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.315437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 178/354] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:05.315437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c456-eacb-4d4e-8fe8-50300db5d1c4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.357944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bdf58-e5a3-47ec-8000-7ccdaf95ec95 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.401207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.402200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.402566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.402902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.404128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:05.405625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:05.409085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 177/355] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:05.409085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.456063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c456-eacb-4d4e-8fe8-50300db5d1c4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.473821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:05.501186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435520-a1ce-4220-b96d-e58278c4c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.501918 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435520-a1ce-4220-b96d-e58278c4c414 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.502266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435520-a1ce-4220-b96d-e58278c4c414 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.502605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435520-a1ce-4220-b96d-e58278c4c414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.506582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:05.540148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e38db-2a98-43a8-b928-ebf987d56c2c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.584571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.585229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.585512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.585785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.586991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e435520-a1ce-4220-b96d-e58278c4c414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:05.588454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e435520-a1ce-4220-b96d-e58278c4c414 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:05.695863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:05.697659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:05.720510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 179/356] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:05.722274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def525b-a044-4a38-9220-351defe13975 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.722761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def525b-a044-4a38-9220-351defe13975 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.723348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def525b-a044-4a38-9220-351defe13975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.742211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4def525b-a044-4a38-9220-351defe13975 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:05.743733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4def525b-a044-4a38-9220-351defe13975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:05.749645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:05.782232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:05.815254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 178/357] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:05.816108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.826524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 180/358] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:05.827622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def525b-a044-4a38-9220-351defe13975 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:05.945236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54885-f779-43a3-9539-d50cc4f8eb48 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.949457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def525b-a044-4a38-9220-351defe13975 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:05.992037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08808ea3-a7e7-4332-a5b8-1939c20a7658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.992608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808ea3-a7e7-4332-a5b8-1939c20a7658 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.993022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:05.994039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:05.994140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808ea3-a7e7-4332-a5b8-1939c20a7658 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.994211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:05.994276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808ea3-a7e7-4332-a5b8-1939c20a7658 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.994340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:05.995158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08808ea3-a7e7-4332-a5b8-1939c20a7658 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:05.996591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08808ea3-a7e7-4332-a5b8-1939c20a7658 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:05.998322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 181/359] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.000602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a994-829e-49c9-ace1-06303e111809 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.000602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a994-829e-49c9-ace1-06303e111809 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.000851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a994-829e-49c9-ace1-06303e111809 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.001931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 182/360] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:06.003902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a994-829e-49c9-ace1-06303e111809 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.008443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.009161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.010855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 179/361] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:05 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.011626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.029067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.052535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.132642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afc67c-59d8-41ae-8494-fd5a1b41514a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.140083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a994-829e-49c9-ace1-06303e111809 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.170266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9306a6-08b1-4ecc-a6fe-a5253f7b37c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.171542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9306a6-08b1-4ecc-a6fe-a5253f7b37c2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.171542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9306a6-08b1-4ecc-a6fe-a5253f7b37c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.171542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9306a6-08b1-4ecc-a6fe-a5253f7b37c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.182157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.182799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.183082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.183354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.184680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.185092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9306a6-08b1-4ecc-a6fe-a5253f7b37c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:06.186154 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.188270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 183/362] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.188880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-0d9306a6-08b1-4ecc-a6fe-a5253f7b37c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:06.190648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:06.191514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:06.192328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:06.192933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 180/363] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:06.193805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.196185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.196185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.213641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:06.214850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:06.295965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.302128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 181/364] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:06.302758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.325093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3731f-c755-4a30-be34-c8445a5e6c33 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.325093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.365770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.366423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.366702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.366962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.368141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.369637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.371511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 184/365] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.372273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.432207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab804ae4-bfd9-4edb-8ffe-a7fd4a7865b2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.476520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.477155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.477511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.477696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.478538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=937d691e04a147e0a4028b930e05179a) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:06.478730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.480149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.486216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 182/366] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.487128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.500823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6f3bd-8d92-4777-a8e9-80db3ccf0152 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.564710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.564710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.564710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.564710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.566658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.568003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.569833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 185/367] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.570612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.583682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.618920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.642202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a0884-2adf-4918-9cb9-10b2309657a0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.695210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.696656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.696656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.696656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.697145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:06.697434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.698818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.714985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 183/368] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.715906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.719832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1273be8-f147-40db-91a2-92cc11fadcbf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.768694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.769433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.769734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.770004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.783543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:06.788307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:06.789264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:06.790044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 186/369] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:06.790703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.867671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a773bf-0f1c-4c6a-879e-eb6efb3c1496 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.898323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.913933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.914551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.915281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.915281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.915985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=937d691e04a147e0a4028b930e05179a) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:06.916255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.917815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.918995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:06.923993 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af41ca-da53-4b26-8095-13455ab10c88 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:06.925630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 184/370] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.926939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:06.968412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:06.969120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:06.969470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:06.969744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:06.970546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:06.970863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:06.972344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:06.978955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 187/371] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:06.980597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:07.069305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d991d35-f835-4611-b22f-6c77a193dd8c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.148237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:07.148794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:07.149103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:07.149383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:07.179977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244649a-1b57-45c1-a1fe-e54a5765bc93 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.232640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:07.233712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:07.234829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:07.235313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:07.236464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:07.236969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:07.238724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:07.251104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 188/372] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:07.257928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:07.259214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:07.274782 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:07.276839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:07.294434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:07.389508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1822eef-576b-49a7-8195-c05ce04e8450 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.412700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 185/373] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:07.422466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:07.427650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:07.428569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:07.429105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:07.429578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:07.501654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:07.503574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:07.533387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:07.592398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:07.618909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fb321-8b2f-483d-92eb-3acc0f7b05c6 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.635653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 189/374] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:07:07.641954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:07.705928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:07.706681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:07.706966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:07.707238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:07.713806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:07.715266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:07.717132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 186/375] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:07 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:07.726735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:07.794015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f923e-1c42-427b-8745-495c34a943c2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.825443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:07.831877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:07.832520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:07.832803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:07.833101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] PATH_INFO: `/v3/projects/55c6cff78b754392afc788fabbf074a6` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:07.837771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] RBAC: Authorizing `identity:get_project(project_id=55c6cff78b754392afc788fabbf074a6)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:07.839426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:07.840874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 190/376] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:07 2022] GET /identity/v3/projects/55c6cff78b754392afc788fabbf074a6 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:07.853307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:07.924783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa012ae-c2c4-4f45-a9b4-6c2d731fd38a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.971110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646d8b9-330e-42c8-ba62-00f1989d2a2a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:07.993690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:07.994338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:07.994617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:07.994889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:07.996045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:07.997512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:07.999378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 187/377] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:07 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:08.019073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:08.019717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:08.020000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:08.020277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] PATH_INFO: `/v3/groups/edded6c4f8b541d69e865a2f747db4b8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:08.022233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:08.027199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] RBAC: Authorizing `identity:get_group(group_id=edded6c4f8b541d69e865a2f747db4b8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:08.029008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:08.031591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 191/378] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:07 2022] GET /identity/v3/groups/edded6c4f8b541d69e865a2f747db4b8 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:08.037034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:08.075737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:08.100350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:08.153987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbf70f-05e0-4fb6-b0b4-bd36d01102ee None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:08.188808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6c078-f2e0-4c59-bf39-1a59af9471b1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:08.202916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:08.204257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:08.204420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:08.207882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:08.207882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] RBAC: Adding query filter params (group.id=edded6c4f8b541d69e865a2f747db4b8, role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=55c6cff78b754392afc788fabbf074a6) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:08.207882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:08.207882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91f5e-ef00-454a-9c65-95aefefd45c2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:08.213772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 192/379] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:08 2022] GET /identity/v3/role_assignments?group.id=edded6c4f8b541d69e865a2f747db4b8&scope.project.id=55c6cff78b754392afc788fabbf074a6&role.id=dd76871a868143b482c4982027fdc1f5 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:08.240972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:08.241636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:08.241918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:08.242191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:08.242932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:08.243237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:08.244695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb121ac-f9a5-4033-a88c-4ab4ff0ea940 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:08.251972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 188/380] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:08 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:08.724755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db061604-760f-451c-af00-325c7b2bbb29 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:08.725111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db061604-760f-451c-af00-325c7b2bbb29 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:08.725349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db061604-760f-451c-af00-325c7b2bbb29 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:08.726703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 193/381] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:08.732208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:08.732679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:08.732764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:08.762022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:08.763225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:08.855600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 189/382] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:08.885804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db061604-760f-451c-af00-325c7b2bbb29 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.042626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db061604-760f-451c-af00-325c7b2bbb29 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.065256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:09.080750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5158ba7-d552-46d1-a2d3-bd2934b22b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.091853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5158ba7-d552-46d1-a2d3-bd2934b22b43 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.091853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5158ba7-d552-46d1-a2d3-bd2934b22b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.091853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5158ba7-d552-46d1-a2d3-bd2934b22b43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.092712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5158ba7-d552-46d1-a2d3-bd2934b22b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e5158ba7-d552-46d1-a2d3-bd2934b22b43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:09.097711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:09.098644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:09.099479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 194/383] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:09.106254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.136582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:09.239876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3767a7-3c42-467f-bc14-c0b2b73baca7 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.256836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b173b5-6f2d-4c29-be71-655a7771607c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.257516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b173b5-6f2d-4c29-be71-655a7771607c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.257516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b173b5-6f2d-4c29-be71-655a7771607c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.258997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 195/384] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:09.272121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.272409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.272630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.279868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.280765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.281484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.281977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.282885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:09.283409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:09.285031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:09.294675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 190/385] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:09.300949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.315492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:09.316655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:09.388611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:09.408836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2c9d-12b5-482c-bc63-570afe4b0cfc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.444635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.444974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 196/386] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:09.446039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.446544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.447019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.448372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:09.450171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:09.452185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 191/387] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:09 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:09.458023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.465829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.576942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c994de-02e6-4e86-a432-e99eeab9d95c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.651815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.652763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.653260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.653702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.668221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:09.676356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:09.677393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:09.678600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 192/388] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:09.684567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.700152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d86c-4fae-418d-9dde-69b216957b43 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.712164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:09.716601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:09.780943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.782814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.783305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.783781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.785902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:09.787635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:09.789590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 197/389] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:09 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:09.795516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.811746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5865ab8-63c3-4943-b57d-e0c1126ee08a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.850548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.851089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.851376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.851653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.852331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:09.852664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:09.854043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:09.867825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 193/390] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:09.869174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:09.938288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da373-cfff-480d-b2d3-b18e44978fc0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.946120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:09.985221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:09.986035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:09.986321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:09.986702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a/groups/833767b73c9f47caa618aac3a45ca2c3/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:09.988711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=937d691e04a147e0a4028b930e05179a, group_id=833767b73c9f47caa618aac3a45ca2c3, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:09.990052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2b20-c6d9-4645-a5f6-bc34a76491d1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:09.990162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:10.025222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26e40a-9451-4523-9cc1-190f63022809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:10.025852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e40a-9451-4523-9cc1-190f63022809 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.026137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e40a-9451-4523-9cc1-190f63022809 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.026418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e40a-9451-4523-9cc1-190f63022809 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.027832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26e40a-9451-4523-9cc1-190f63022809 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:10.029296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26e40a-9451-4523-9cc1-190f63022809 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:10.030864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 194/391] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:09 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:10.034439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a2b4-9fbf-4e00-8944-b607020edaf9 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.034439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a2b4-9fbf-4e00-8944-b607020edaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.034439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a2b4-9fbf-4e00-8944-b607020edaf9 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.034439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 195/392] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:10.039507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.039798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.040075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.061039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 198/393] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 10 13:07:09 2022] PUT /identity/v3/projects/937d691e04a147e0a4028b930e05179a/groups/833767b73c9f47caa618aac3a45ca2c3/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:07:10.061039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:10.079982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:10.081226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:10.136944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:10.231665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 196/394] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:10.254932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93820567-1042-459d-b1ad-cfe994dc30b0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:10.282945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:10.301486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:10.302141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.302561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.302902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.304858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:10.306744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:10.309135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 199/395] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:10 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:10.316254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:10.374264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:10.411423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d109-7208-4c0a-a393-7a05c1752e35 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:10.453392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:10.454539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.454977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.455400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.467205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:10.473656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:10.474713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:10.475667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:10.477779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 197/396] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:10.484375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:10.551458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e5a2-fd2a-4565-8adf-d730ef0bf3ef None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:10.566345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:10.632777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b8ff-3ad7-41cc-9aec-8b66ab167e16 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:10.653739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdafa80-d247-40b5-8d70-f1c1de207432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:10.654432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdafa80-d247-40b5-8d70-f1c1de207432 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.654709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdafa80-d247-40b5-8d70-f1c1de207432 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.654983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdafa80-d247-40b5-8d70-f1c1de207432 None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a/groups/833767b73c9f47caa618aac3a45ca2c3/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.666437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdafa80-d247-40b5-8d70-f1c1de207432 None admin] RBAC: Authorizing `identity:create_grant(project_id=937d691e04a147e0a4028b930e05179a, group_id=833767b73c9f47caa618aac3a45ca2c3, role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:10.667819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdafa80-d247-40b5-8d70-f1c1de207432 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:10.687700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:10.694311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:10.695155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.695606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.696025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.696845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:10.697319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:10.699115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:10.709373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 198/397] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:10.717115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:10.752205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 200/398] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 10 13:07:10 2022] PUT /identity/v3/projects/937d691e04a147e0a4028b930e05179a/groups/833767b73c9f47caa618aac3a45ca2c3/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:10.800909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e19c-2010-4990-b129-558146117b07 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.801275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e19c-2010-4990-b129-558146117b07 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.801486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e19c-2010-4990-b129-558146117b07 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.803335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 201/399] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:10.808201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084dc05-5c40-4a73-bbe5-29e99da3d968 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.808481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084dc05-5c40-4a73-bbe5-29e99da3d968 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.808753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084dc05-5c40-4a73-bbe5-29e99da3d968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.838307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a084dc05-5c40-4a73-bbe5-29e99da3d968 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:10.839555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a084dc05-5c40-4a73-bbe5-29e99da3d968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:10.876301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a72d-24b2-49f0-8daa-7dab30833e6c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:10.922667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 202/400] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:10.923316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7d935-173b-4c6e-b31f-b0275c3acfef None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.923605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7d935-173b-4c6e-b31f-b0275c3acfef None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.923876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7d935-173b-4c6e-b31f-b0275c3acfef None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.925577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 203/401] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:10.930522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.930807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.931100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.934947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:10.935873 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:10.936301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:10.936723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:10.938863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:10.940484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:10.945323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 199/402] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:10 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:10.946183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:10.952576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:10.953902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:11.018126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.044648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 204/403] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:11.045318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.063999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d01680-34f5-450e-822d-bae4870acbb0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.080695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.102186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.103206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.103707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.104196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.115566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:11.122145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:11.123017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:11.123825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:11.127106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 200/404] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:11.128095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.167766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b26603-e941-4510-9e1f-e1c266897f39 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.207573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.208897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.209494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.210122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.211512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.213277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.220692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 205/405] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:11.221815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.263290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e4f4-cdae-4e27-94dd-2599ad57066d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.291774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.310165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.310807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.311086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.311596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.325220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:11.326149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:11.332974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:11.333945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:11.335092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 201/406] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:11.335917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.337405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1e248-26be-462a-a5f6-a6e430dc5339 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.378380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.379013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.382265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.382265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.382265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:11.382265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.382265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.393523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 206/407] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:11.394342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.455554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d1ec-211d-48d9-b63a-57360413dfea None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.498382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.499515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.500476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.501103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/groups/833767b73c9f47caa618aac3a45ca2c3/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.503589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] RBAC: Authorizing `identity:create_grant(project_id=5765aa65f6dd4dac9f062fb2ffa7a762, group_id=833767b73c9f47caa618aac3a45ca2c3, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.505157 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.519039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976e453-447b-4cc2-9e8e-049e8d77b0c4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.554809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.555505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.555886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.556210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.556931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:11.557329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.558776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.571791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 207/408] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:11.571791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.592154 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.613218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 202/409] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 10 13:07:11 2022] PUT /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/groups/833767b73c9f47caa618aac3a45ca2c3/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:11.614516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.693437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.726606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e97ee-5aa9-4467-a812-45aec6f6a52f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.781577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.782225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.782503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.783264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] PATH_INFO: `/v3/groups/edded6c4f8b541d69e865a2f747db4b8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.784444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] RBAC: Authorizing `identity:get_group(group_id=edded6c4f8b541d69e865a2f747db4b8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.785937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.788222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 208/410] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/groups/edded6c4f8b541d69e865a2f747db4b8 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:11.793756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.826781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c5224-ea98-4d2b-b313-63256d48d421 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.871970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.872723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.873136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.873412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.875158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.876621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.880000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 203/411] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:11.881323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:11.883999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:11.897772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d888c7-41fd-4d0d-85a1-c886cebe951a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:11.932008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:11.932642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:11.932920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:11.933230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] PATH_INFO: `/v3/projects/55c6cff78b754392afc788fabbf074a6` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:11.934799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] RBAC: Authorizing `identity:get_project(project_id=55c6cff78b754392afc788fabbf074a6)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:11.935891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:11.938247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 209/412] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/projects/55c6cff78b754392afc788fabbf074a6 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:11.943745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:12.048778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cb131-02e7-4bc2-bdae-17ac53609d94 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:12.080049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b042b16-f68b-4412-80a9-b98ed5aae3ae None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:12.105357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:12.126762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0a4a8-effe-45ed-9ed5-250384ec67f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:12.127446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a4a8-effe-45ed-9ed5-250384ec67f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.127726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a4a8-effe-45ed-9ed5-250384ec67f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.127997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a4a8-effe-45ed-9ed5-250384ec67f5 None admin] PATH_INFO: `/v3/projects/55c6cff78b754392afc788fabbf074a6/groups/edded6c4f8b541d69e865a2f747db4b8/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.134338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0a4a8-effe-45ed-9ed5-250384ec67f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=55c6cff78b754392afc788fabbf074a6, group_id=edded6c4f8b541d69e865a2f747db4b8, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:12.135962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0a4a8-effe-45ed-9ed5-250384ec67f5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:12.138239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:12.139083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.139540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.140041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.143149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:12.145483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:12.147530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 204/413] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:11 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:12.153700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:12.247291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 210/414] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 10 13:07:11 2022] PUT /identity/v3/projects/55c6cff78b754392afc788fabbf074a6/groups/edded6c4f8b541d69e865a2f747db4b8/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:12.258955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345afd7-7d32-4bf9-bbf8-448dd72f4799 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:12.299636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cccdc-17f1-400c-8a56-d39de43413c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:12.300296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cccdc-17f1-400c-8a56-d39de43413c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.300581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cccdc-17f1-400c-8a56-d39de43413c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.300854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cccdc-17f1-400c-8a56-d39de43413c1 None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/groups/833767b73c9f47caa618aac3a45ca2c3/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.303188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cccdc-17f1-400c-8a56-d39de43413c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5765aa65f6dd4dac9f062fb2ffa7a762, group_id=833767b73c9f47caa618aac3a45ca2c3, role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:12.304724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cccdc-17f1-400c-8a56-d39de43413c1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:12.387530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 205/415] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 10 13:07:12 2022] PUT /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762/groups/833767b73c9f47caa618aac3a45ca2c3/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:12.540005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa0255-aa49-4291-9a13-4299d59250f7 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.540544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa0255-aa49-4291-9a13-4299d59250f7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.540544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa0255-aa49-4291-9a13-4299d59250f7 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.541965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 211/416] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:12.547920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.548238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.548489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.577281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:12.579217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:12.624468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:12.678841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 206/417] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:12.705916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa0255-aa49-4291-9a13-4299d59250f7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:12.728109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:12.853663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa0255-aa49-4291-9a13-4299d59250f7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:12.895266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342d8a-0f62-41eb-93fc-29fc5827939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:12.895897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342d8a-0f62-41eb-93fc-29fc5827939b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.896175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342d8a-0f62-41eb-93fc-29fc5827939b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.896445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342d8a-0f62-41eb-93fc-29fc5827939b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.907373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342d8a-0f62-41eb-93fc-29fc5827939b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-49342d8a-0f62-41eb-93fc-29fc5827939b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:12.912560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:12.913573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:12.914504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 212/418] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:12.922958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:12.973274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65917-279e-405a-b89e-1da63d4b36d6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.973616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65917-279e-405a-b89e-1da63d4b36d6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.973820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65917-279e-405a-b89e-1da63d4b36d6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:12.975155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 213/419] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:12.985359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:12.985627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:12.985899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:13.019833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:13.021063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:13.080893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3fed2-c0c1-4870-b5ed-490bbd712486 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:13.129969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 214/420] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:13.160244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:13.161275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:13.161763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:13.162260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:13.163135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:13.163648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:13.165259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:13.174043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:13.181532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:13.190445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 207/421] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:13.195842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:13.250347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:13.330212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa15d1-099a-454d-a4a0-9bf58dbb968a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:13.367240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:13.368043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:13.368431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:13.369217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:13.428748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e9a5-cf84-4bd5-8c4c-c674611dff89 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:13.447729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:13.448542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:13.494696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:13.525999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:13.526906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:13.527191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:13.527465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:13.554287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:13.559177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:13.560141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:13.560948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:13.574316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 215/422] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:13.586375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:13.624457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 208/423] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:07:13.627285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:13.675481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:13.716942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccdffe-a279-4c74-a93a-d52bbad475b1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:13.767257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:13.768165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:13.768592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:13.769016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:13.769963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:13.770273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:13.772088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:13.784528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 216/424] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:13.792662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:13.870372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb72fe3-f4f5-4a70-b300-dbbde63b410f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:13.900035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:13.909942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:13.913203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:13.913203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:13.913203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:13.913203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:13.913951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:13.916176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 209/425] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:13 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:13.922271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:13.972655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38edfaa-ca30-4e9a-85cc-8c44b939bfae None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.003327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.012727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707edab-c64b-45b6-a139-29254787f232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.013372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707edab-c64b-45b6-a139-29254787f232 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.013657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707edab-c64b-45b6-a139-29254787f232 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.013928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707edab-c64b-45b6-a139-29254787f232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.097169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707edab-c64b-45b6-a139-29254787f232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.099422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707edab-c64b-45b6-a139-29254787f232 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.152451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f68a-2f06-4779-a2bf-4b4749b4331d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.192139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e1e27-0e21-4d8d-84f8-83237ba2c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.193192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e1e27-0e21-4d8d-84f8-83237ba2c250 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.193961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e1e27-0e21-4d8d-84f8-83237ba2c250 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.194475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e1e27-0e21-4d8d-84f8-83237ba2c250 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.195850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e1e27-0e21-4d8d-84f8-83237ba2c250 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.197643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e1e27-0e21-4d8d-84f8-83237ba2c250 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.199955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 210/426] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:13 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:14.201556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b01de5-fd45-4dd7-9886-4c44e522c573 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.202063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b01de5-fd45-4dd7-9886-4c44e522c573 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.202533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b01de5-fd45-4dd7-9886-4c44e522c573 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.204676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 211/427] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:14.209733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b01de5-fd45-4dd7-9886-4c44e522c573 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.219849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 217/428] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:14.221046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.221328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.221603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.239291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:14.240497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:14.261823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.313445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.317104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 218/429] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:14.318035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.324396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b01de5-fd45-4dd7-9886-4c44e522c573 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.364048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.365169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.365652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.366122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.367102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=937d691e04a147e0a4028b930e05179a) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:14.367623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.369533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.376408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 212/430] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:14.377593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.444381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69419d6c-972d-479c-bda1-2b80970c2506 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.482299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.483056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.483236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.483501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] PATH_INFO: `/v3/projects/937d691e04a147e0a4028b930e05179a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.485256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] RBAC: Authorizing `identity:get_project(project_id=937d691e04a147e0a4028b930e05179a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.486845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.488789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 219/431] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/projects/937d691e04a147e0a4028b930e05179a => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:14.494670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.502441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fc84a-5636-4273-8fdd-553c3792fafd None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.522160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.537591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.545801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.547699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.548195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.548798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.563789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:14.570753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:14.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:07:14.572529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:14.576051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 213/432] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:14.587589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.640585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858722b-6e9f-487a-98ff-8948d42cd37a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.700361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f839b-7228-4f00-891e-702a27ebe1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.701945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f839b-7228-4f00-891e-702a27ebe1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.701945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f839b-7228-4f00-891e-702a27ebe1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.702202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f839b-7228-4f00-891e-702a27ebe1a9 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.703429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f839b-7228-4f00-891e-702a27ebe1a9 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.705078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f839b-7228-4f00-891e-702a27ebe1a9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.708569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 220/433] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:14.708755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0045102-8568-496e-bfb1-8df38c948568 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.711543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0045102-8568-496e-bfb1-8df38c948568 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.711543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0045102-8568-496e-bfb1-8df38c948568 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.711543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 221/434] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:14.716841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0045102-8568-496e-bfb1-8df38c948568 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.753666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e972c7-7ab2-45ad-ab5e-83821032df26 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.796186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.797988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.798279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.798551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.799234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:14.799541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.800919 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fc982-06d0-4104-89b9-ab25f8e42d20 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.810177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 214/435] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:14.811689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.811689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.811880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.833167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:14.834382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:14.842297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.852970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0045102-8568-496e-bfb1-8df38c948568 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:14.896126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:14.900181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:14.900181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:14.900181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:14.900181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:14.902167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=937d691e04a147e0a4028b930e05179a) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:14.902167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:14.903741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:14.913149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 222/436] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=937d691e04a147e0a4028b930e05179a&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:14.913930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:14.924247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 215/437] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:14.950756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.056349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1ee1c-05f4-44b8-9c4e-a2ce22e69c76 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.089539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241333-bf2e-4d40-8929-0ab8739c7239 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.098546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9be68f-df9c-43d8-9e99-1b2488322aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.099360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9be68f-df9c-43d8-9e99-1b2488322aaf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.099797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9be68f-df9c-43d8-9e99-1b2488322aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.100219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9be68f-df9c-43d8-9e99-1b2488322aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.128467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.137589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d2e10-f2fa-4fdb-9bcf-d77a35b9de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.138231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d2e10-f2fa-4fdb-9bcf-d77a35b9de60 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.138531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d2e10-f2fa-4fdb-9bcf-d77a35b9de60 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.138808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d2e10-f2fa-4fdb-9bcf-d77a35b9de60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.151545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d2e10-f2fa-4fdb-9bcf-d77a35b9de60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a05d2e10-f2fa-4fdb-9bcf-d77a35b9de60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:15.157637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:15.159124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:15.159926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:15.163041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 216/438] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:15.163041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91942ca-8499-42ac-9635-34b2eba5a80f None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.163041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91942ca-8499-42ac-9635-34b2eba5a80f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.163041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91942ca-8499-42ac-9635-34b2eba5a80f None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.165066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 217/439] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:15.165630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91942ca-8499-42ac-9635-34b2eba5a80f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.179682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9be68f-df9c-43d8-9e99-1b2488322aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.181162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9be68f-df9c-43d8-9e99-1b2488322aaf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.222209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.292457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91942ca-8499-42ac-9635-34b2eba5a80f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.302484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 223/440] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:15.303723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.304016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.304290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.322878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:15.324018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:15.331751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.332367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.332647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.332919 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.333629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:15.333938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.335319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.344260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 218/441] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:15.345222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.378339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.394744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 224/442] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:15.395569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.446134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd70d13-be48-4d6d-81e2-181caf451c26 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.447667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.484609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.485289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.485576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.485851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.486997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.488450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.490129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 219/443] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:15.490887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.503759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616b9e0-4cd7-4101-8d9a-7b5d8a74fb41 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.538337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.538950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.539234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.539507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.589933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dad38e-c75e-4876-8584-541942da76da None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.593028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.605875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.607347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.622983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.623606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.623883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.624154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.635192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:15.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:15.642122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:15.642907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 220/444] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:15.643570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.676906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.710245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 225/445] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:15.711084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.746138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de12a7-cd4f-4fe0-97b5-8d40e51d882a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.780535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.781237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.781521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.781999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.783300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.784715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.786328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 221/446] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:15.787088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.800334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.815917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5feb9-5499-4218-a6f4-a13d5e28d780 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.850625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.851147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.851424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.851700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.852351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:15.852650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.854031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.862634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 226/447] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:15.863211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.896275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:15.900815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f77d-b14b-4e3c-8b4f-252b7d9a9308 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:15.934329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:15.934969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:15.935455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:15.935941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] PATH_INFO: `/v3/projects/55c6cff78b754392afc788fabbf074a6` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:15.938051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] RBAC: Authorizing `identity:get_project(project_id=55c6cff78b754392afc788fabbf074a6)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:15.939888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:15.941763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 222/448] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/projects/55c6cff78b754392afc788fabbf074a6 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:15.942332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:15.967201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205c8b5-81b2-4142-ba29-cef1de5f771d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.001810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.002417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.002699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.002973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.003700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=31c6a032fe034fd5abc43cc4ce065209, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:16.004011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.005468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.011471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 227/449] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=31c6a032fe034fd5abc43cc4ce065209 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:16.012340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:16.039433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:16.047757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e158e7-6e21-494c-9711-e5a30952597c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.088811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.089484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.089764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.090035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.128683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:16.148293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0829c-3195-4703-8f73-e7a21c7b27c9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.169673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.171152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.199988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.200610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.200891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.201229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] PATH_INFO: `/v3/groups/edded6c4f8b541d69e865a2f747db4b8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.202370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] RBAC: Authorizing `identity:get_group(group_id=edded6c4f8b541d69e865a2f747db4b8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.205838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.207460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 228/450] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:16 2022] GET /identity/v3/groups/edded6c4f8b541d69e865a2f747db4b8 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:16.214559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:16.290943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 223/451] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:16.296314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:16.303296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:16.323664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87f9df-4d8c-4a9b-bbcc-5a515cdc1d8e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.361443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.362090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.362720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.362720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.363338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] RBAC: Adding query filter params (group.id=edded6c4f8b541d69e865a2f747db4b8, role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=55c6cff78b754392afc788fabbf074a6) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:16.363642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.365098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.370880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 229/452] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:16 2022] GET /identity/v3/role_assignments?group.id=edded6c4f8b541d69e865a2f747db4b8&scope.project.id=55c6cff78b754392afc788fabbf074a6&role.id=dd76871a868143b482c4982027fdc1f5 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:16.376381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:16.417545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1eea-3a03-47b7-9548-9a893c76b513 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.455059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.455708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.455988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.456258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] PATH_INFO: `/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.457505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] RBAC: Authorizing `identity:get_project(project_id=5765aa65f6dd4dac9f062fb2ffa7a762)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.458988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.460658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 224/453] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:07:16 2022] GET /identity/v3/projects/5765aa65f6dd4dac9f062fb2ffa7a762 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:16.466321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:16.569788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec965-82f5-4138-bcee-2abb23b1351d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.603040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4402-d2b8-40d2-8560-a1867be74e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.603670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4402-d2b8-40d2-8560-a1867be74e3a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.603949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4402-d2b8-40d2-8560-a1867be74e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.604222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4402-d2b8-40d2-8560-a1867be74e3a None admin] PATH_INFO: `/v3/groups/833767b73c9f47caa618aac3a45ca2c3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.605378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b4402-d2b8-40d2-8560-a1867be74e3a None admin] RBAC: Authorizing `identity:get_group(group_id=833767b73c9f47caa618aac3a45ca2c3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.606802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b4402-d2b8-40d2-8560-a1867be74e3a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.608376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 230/454] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 10 13:07:16 2022] GET /identity/v3/groups/833767b73c9f47caa618aac3a45ca2c3 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:16.614370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:16.714959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83cea-bc86-445b-a155-373b5c498f9f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:16.747199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:16.747790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:16.748065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:16.748335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:16.749074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] RBAC: Adding query filter params (group.id=833767b73c9f47caa618aac3a45ca2c3, role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:16.749380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:16.751069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5029c-e37e-4983-acca-34fb81c57982 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:16.752755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:16.757034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 225/455] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 10 13:07:16 2022] GET /identity/v3/role_assignments?group.id=833767b73c9f47caa618aac3a45ca2c3&scope.project.id=5765aa65f6dd4dac9f062fb2ffa7a762&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:19.448884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d08ab6-6a00-49c9-860f-60a02bf2c401 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.449415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff962-cae0-4a4d-b7c3-9192c58926d7 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.449415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff962-cae0-4a4d-b7c3-9192c58926d7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.449800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff962-cae0-4a4d-b7c3-9192c58926d7 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.450398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d08ab6-6a00-49c9-860f-60a02bf2c401 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.450911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 226/456] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:19.451296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d08ab6-6a00-49c9-860f-60a02bf2c401 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.453156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 231/457] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:19.458481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.458614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.458829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.459047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.459480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.459937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.484762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:19.486323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:19.489996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:19.491359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:19.501428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:19.599115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 227/458] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:19.614268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 232/459] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:19.623196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:19.653228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:19.771328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25911f36-3ee6-433d-b442-deff99763de4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:19.775123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd81dd-3f6f-42d1-8f67-05340473d1d5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:19.808112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:19.812408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:19.813303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.813727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.814143 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.817722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:19.818458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.818832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.819267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.825689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:19.828188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:19.830195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:19.833281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:19.834797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7e48c176-ce62-417e-981e-443e0751a75f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:19.845712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:19.846658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:19.847509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:19.847509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:19.847509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:19.849684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 233/460] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:19.851052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca70c0-554d-4c88-9df8-947df0eec106 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:19.851496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca70c0-554d-4c88-9df8-947df0eec106 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:19.854756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca70c0-554d-4c88-9df8-947df0eec106 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:19.854756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 234/461] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:19.855398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca70c0-554d-4c88-9df8-947df0eec106 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:19.855594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:19.856454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:19.857319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:19.857319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:19.857319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:19.857319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 228/462] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:19.866102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:19.979211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca70c0-554d-4c88-9df8-947df0eec106 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:19.992559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15b2c8-a264-47a3-b2d7-704d8d176b9f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.023509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.024455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.024940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.025445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.026314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:20.027083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.028812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba4b20-a9be-4401-9b0b-eb8760466095 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.039391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.040288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.040574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.040852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.041539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:20.041854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.043238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef82bc8-0891-4361-b676-e10cba89ccd1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.043664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 235/463] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:20.045274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.045561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.045835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.065299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:20.066494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.070270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 229/464] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:20.071519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b669728-0c27-4e6f-9d2e-b822887a9f0d None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.071810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b669728-0c27-4e6f-9d2e-b822887a9f0d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.072089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b669728-0c27-4e6f-9d2e-b822887a9f0d None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.073346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 230/465] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:20.074710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b669728-0c27-4e6f-9d2e-b822887a9f0d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.118653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.130448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.139044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 236/466] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:20.140049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.200851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b669728-0c27-4e6f-9d2e-b822887a9f0d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.238677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f0e96-481c-420b-b039-195c102ece3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.239277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0e96-481c-420b-b039-195c102ece3d None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.239607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0e96-481c-420b-b039-195c102ece3d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.239882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0e96-481c-420b-b039-195c102ece3d None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.240516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f0e96-481c-420b-b039-195c102ece3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.241965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f0e96-481c-420b-b039-195c102ece3d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.249353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f59443-9eed-41d6-8624-57427f97ff0b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.283874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.284404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.284678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.284944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.285602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.287018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.329915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 231/467] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/users => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:20.331361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.331719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.332071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.352320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:20.353705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.417406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 237/468] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:20.419765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.471543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 232/469] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:20.497293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.506979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.547525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.577410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9e183-5eab-4f2a-a403-bc77378c86a7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.619948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.620594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.620876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.621230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.638935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.642246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.666276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4923a8-ac13-495b-85e4-6b1dc2a40beb None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:20.670539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:20.676071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:20.676900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:20.676900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:20.676900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:20.676900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 238/470] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:20.684392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.707985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.708684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.708966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.709263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.721656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.723319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:20.740164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:20.741176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:20.742529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:20.742529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:20.742529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:20.742529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 233/471] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:20.748698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.783528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.792372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b76f9-e0ee-450f-9986-d4c09f948f6a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.828944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.829523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.829773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.830048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.830693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:20.831027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.832517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaff87c-ae7a-4e56-aac8-b05b807acf3e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.843939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.849622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 239/472] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:20.850902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b856b3-3265-42f0-a0b8-8cc8b454c4ec None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.851191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b856b3-3265-42f0-a0b8-8cc8b454c4ec None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.851464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b856b3-3265-42f0-a0b8-8cc8b454c4ec None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.852820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 240/473] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:20.858470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.858642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.858938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.877444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a868940-723c-4075-9c72-73e272e663ad None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.882132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:20.883339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.916482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.917260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.917545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.917829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.918515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:20.918820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.920223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.932754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 234/474] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:20.933662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.970962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 241/475] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:20.972182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:21.059245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4982f8b-7699-4d95-a32f-89695038b5bd None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:21.079746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:21.098984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded351-4310-4cbe-ba6c-ba510948965b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:21.104746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:21.105390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:21.105673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:21.105946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:21.106585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:21.108074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:21.122916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:21.142102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:21.142840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:21.143243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:21.143647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:21.144375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:21.145977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:21.254627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 242/476] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/users => generated 249 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:21.254627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:21.255516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 235/477] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:07:20 2022] POST /identity/v3/users => generated 248 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:21.417819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39381b-2325-4a0d-a3b2-9022fa35da0d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:21.457191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:21.458097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:21.458570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:21.459001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:21.459872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:21.461536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:21.512411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.sql.core [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 13:07:21.512411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 13:07:21.512411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [parameters: {'id': '2454eb17ade84e6f843edef13d309444', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 13:07:21.512411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80228) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result.read() Aug 10 13:07:21.541729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:07:21.542835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application yield resource Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise self.value Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application session.commit() Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 13:07:21.543935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.session.flush() Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application compat.raise_( Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 13:07:21.545020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:07:21.546089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result.read() Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:07:21.547822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [parameters: {'id': '2454eb17ade84e6f843edef13d309444', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:21.548864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 13:07:21.550235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:21.553051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 243/478] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 10 13:07:21 2022] POST /identity/v3/roles => generated 144 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:21.559675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:21.711360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef030e45-3f92-4ea7-9104-1fa8e9748c68 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:21.755749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c46d6-0e5f-4cab-97b2-c4c4942905cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:21.756653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c46d6-0e5f-4cab-97b2-c4c4942905cd None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:21.757097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c46d6-0e5f-4cab-97b2-c4c4942905cd None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:21.757514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c46d6-0e5f-4cab-97b2-c4c4942905cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:21.776298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c46d6-0e5f-4cab-97b2-c4c4942905cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-561c46d6-0e5f-4cab-97b2-c4c4942905cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:21.782887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:21.783942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:07:21.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:21.787192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 236/479] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:21.792800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:21.799556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:21.934754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be393e4-c4a1-4c5f-bb24-311d9f64f5c5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:21.976847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:21.977405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:21.977687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:21.977970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:21.978626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:21.978926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:21.980311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5cfc7-c192-4f91-b866-5af69f797045 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:21.991084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 244/480] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:22.887412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb0228-a38a-48c1-83dd-b891efdd8963 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:22.888415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb0228-a38a-48c1-83dd-b891efdd8963 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:22.888925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb0228-a38a-48c1-83dd-b891efdd8963 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:22.891299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 237/481] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:22.896595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f012bb-d02f-41f2-9b59-2f097edff916 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:22.897458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f012bb-d02f-41f2-9b59-2f097edff916 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:22.897951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f012bb-d02f-41f2-9b59-2f097edff916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:22.922496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d7f012bb-d02f-41f2-9b59-2f097edff916 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:22.923997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d7f012bb-d02f-41f2-9b59-2f097edff916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:23.008717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:23.019575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 245/482] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:23.033503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fbbf-f5a8-4870-8d22-5c6ff15c0af5 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.034063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fbbf-f5a8-4870-8d22-5c6ff15c0af5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.034557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fbbf-f5a8-4870-8d22-5c6ff15c0af5 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.036466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 238/483] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:23.049278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.049278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.049278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.050076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fbbf-f5a8-4870-8d22-5c6ff15c0af5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.064089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:23.067635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:23.137713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:23.188730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 246/484] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:23.204024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fbbf-f5a8-4870-8d22-5c6ff15c0af5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:23.223366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.239467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:23.240100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.240380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.240650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.251852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:23.256818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:23.257733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:23.258549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 239/485] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:23.278955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.387697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b78b-41ea-41ca-a48d-a9884287e227 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:23.405536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:23.418898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62bbce-9db7-4f90-97d6-ce24bd192a04 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:23.434177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:23.434790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.435064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.435376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.436044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:23.436342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:23.438020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:23.458046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 240/486] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:23.464575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.475747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:23.494304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:23.495086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.495293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.495565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.532229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:23.537014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:23.537890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:23.538689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:23.542631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 247/487] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:23.545969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.584052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a2de-42b3-44e3-8413-64f7c759cd26 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:23.620262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:23.620946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.621289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.621572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.632890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:23.634534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:23.655456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:23.656869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:23.657752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:23.657752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:23.657752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:23.657752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 241/488] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:23.664060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.723558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:23.772405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac17b9-3567-4e43-8c3b-9c7635e41c8d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:23.814025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96042c-4b28-47d4-942c-50edfc00b229 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:23.821745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:23.823102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.823972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.824849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.826286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:23.827092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:23.830951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d706d-2100-4645-957d-daabeed8ba28 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:23.848224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 248/489] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:23.851025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9099d-dada-447e-967f-be45c2c7d033 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.851902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9099d-dada-447e-967f-be45c2c7d033 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.852174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:23.852778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.853073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.853350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.853834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9099d-dada-447e-967f-be45c2c7d033 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.853984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:23.854269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:23.855798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3b675-c05e-4bef-b30a-eb230a118024 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:23.857339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 249/490] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:23.859163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9099d-dada-447e-967f-be45c2c7d033 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.873395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 242/491] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:23.878980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:23.878980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:23.878980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:23.894645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:23.896030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:23.964119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:23.967417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 243/492] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:23.973859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:23.998002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.053379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9099d-dada-447e-967f-be45c2c7d033 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.087558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.088185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.088466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.088737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.096698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10714914-3cd4-4469-ac42-841e71eeb1a3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.099584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.101040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-779e4d68-029e-4259-8c49-0ce9806979f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:24.116120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:24.117329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:24.118229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:24.118229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:24.118229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:24.121813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 250/493] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:24.122920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5fc93-4cf0-4f89-932d-0b8120e14e29 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.123203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5fc93-4cf0-4f89-932d-0b8120e14e29 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.123536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5fc93-4cf0-4f89-932d-0b8120e14e29 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.125505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 251/494] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:24.129958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5fc93-4cf0-4f89-932d-0b8120e14e29 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.134322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.134940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.135222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.135495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.143050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.144497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:24.158822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:24.159728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:24.160615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 244/495] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:07:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:24.161212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.249255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.252131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.256893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5fc93-4cf0-4f89-932d-0b8120e14e29 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.276048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58efdee-a459-41ba-9dd7-760589fc85b1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.291570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996d52c-529e-49d5-ba77-fa9d94d95f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.298332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996d52c-529e-49d5-ba77-fa9d94d95f8a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.298332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996d52c-529e-49d5-ba77-fa9d94d95f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.298332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996d52c-529e-49d5-ba77-fa9d94d95f8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.304588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996d52c-529e-49d5-ba77-fa9d94d95f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9996d52c-529e-49d5-ba77-fa9d94d95f8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:24.309292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:24.310408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:24.311334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 252/496] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:24.314218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.314945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.316357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.318641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.319258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.319601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.319822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.320473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:24.320779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.322189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.341100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:24.341307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 245/497] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:24.341307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.341475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:24.415972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 253/498] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:24.425172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.488529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5f067-66f6-46f5-a88e-442da00bf9f8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.526005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.526617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.527164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.527436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.528164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:24.528467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.529946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.539527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.551720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac33cd9-c160-4c96-884a-c873c6ab1384 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.557293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.582784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 246/499] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:24.582784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.593724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.594341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.594621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.594896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.595567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:24.595897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.597257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.605328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 254/500] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:24.610707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.687866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac52a-f25f-4b21-aa04-485489376197 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.722025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740a506-b208-4d04-926b-a413e07ae646 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.731858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.732669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.732946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.733252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.740568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.742048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:24.753625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:24.754578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:07:24.755467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:24.756138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.756222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 247/501] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:24.756288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.756288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.756412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.756475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.769690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:24.774796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:24.775716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 255/502] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:24.776626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.792500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.820896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:24.865630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40a3b9-0cbe-482c-9f5b-cff1aa551368 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.878494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab85c50-8b06-4869-bfc3-619c62809834 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:24.898878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.899514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.899798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.900070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.912474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:24.913126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.913409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.913684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.923950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.925411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4bd13d6b-8c2c-44f7-a1b8-5e42201902ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:24.939269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:24.940343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:24.941428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:24.941428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:24.941428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:24.941428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 256/503] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:24.941659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddaa0a-159e-4c06-8a0f-bc6ba67f27e8 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:24.941781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddaa0a-159e-4c06-8a0f-bc6ba67f27e8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:24.942055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddaa0a-159e-4c06-8a0f-bc6ba67f27e8 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:24.943270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 257/504] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:24.943958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddaa0a-159e-4c06-8a0f-bc6ba67f27e8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:24.975363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:24.976839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.006721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.031540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.045496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddaa0a-159e-4c06-8a0f-bc6ba67f27e8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.074529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 248/505] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:25.075357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.078694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.079423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.079712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.080039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.080773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:25.081112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.082550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.119256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 258/506] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 10 13:07:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:25.120180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.175533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d124c20-65cd-45c7-a7ef-d098b96b0862 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.207932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.208525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.208950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.209278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.209937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:25.210239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.211585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad39a3-b74c-4837-a936-01fe836fc2aa None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.219854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 249/507] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:25.220197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac17ed-c56f-404b-b9a3-3f98120511b0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.220804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.221121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.221394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.238204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:25.239415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:25.241581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.253020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.255433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.256034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.256314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.256584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.263046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:25.263046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.263046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.269785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 259/508] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:25.270566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.305954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 250/509] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:25.307001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.372353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb2ebe-9abf-447a-bb00-fcbf1881af77 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.406188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.406809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.407081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.407346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.410323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e27dc7-c6cc-4114-bef8-f4dce9cdf12e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.417702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.419133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:25.433732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:25.434834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:25.435684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:25.435684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:25.435684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:25.435684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 260/510] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:25.435684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.448056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.448670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.448950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.449255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.466833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.486253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.518050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.519521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.552888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd41ff-c8b2-46fd-bdd9-2a80ff66c831 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.585985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.586539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.586830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.587101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.597559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.599011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:25.613081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:25.614112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:25.619874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:25.619874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:25.619874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:25.619874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 261/511] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:25.619874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.620194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 251/512] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:25.620194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.666751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.710776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.717241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de22c-ba39-4cea-a727-ab407a0914fb None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.726145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0e4a-d5b2-4805-a2d7-aefc94d62ad7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.751084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.751799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.752154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.752424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.760073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.760696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.760977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.761282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.761998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.763442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.769176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.770606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:25.778179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:25.779259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:25.780120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:25.780120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:25.780120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:25.780120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 252/513] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:25.780463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:25.783115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:25.783995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 262/514] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:25.785309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.883048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d17a9-55a6-4d2c-9828-88cab99e9f92 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.884790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.887971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96399-f0e9-4ede-ad17-92d0535c582b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:25.915888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.916484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.916769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.917090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.917729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:25.918030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.919396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.923003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:25.923610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:25.923894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:25.924168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:25.924811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:25.925149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:25.926504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:25.932255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 263/515] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:25.933076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:25.936314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:25.946605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 253/516] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:25.946605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.031552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f4864-de86-4809-a0c7-ed8f102b8bf2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.044281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658863f-44cb-40c6-9a40-ddb89fa59a57 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.064655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.065311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.065591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.065862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.075554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.077007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.078364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.078968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.079248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.079514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.080147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:26.080444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.081812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:26.091785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:26.092921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:26.094069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:26.094069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:26.094069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:26.094286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 254/517] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:26.094366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 264/518] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:26.094366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.097866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.119117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.150195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.194626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a24c2-632c-4919-9e74-6a5acaf4f2ce None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.196915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbb59f-4816-48d0-8676-d491b9e9b386 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.228007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.228632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.228912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.229207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.231068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.232169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.232461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.232738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.233498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:26.233809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.235253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.236588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.238102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:26.248799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:26.250298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:26.251197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 265/519] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:26.252284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.253023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 255/520] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:26.253649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.331635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.351986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950511-14dc-4ef8-a7e3-476fbaaf5999 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.353376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81e53f-f200-4c32-87af-6a7c0a626788 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.369399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.386904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.387343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.387532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.387815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.387946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.388136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.388225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.388491 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.389163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.391250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.395331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.396799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:26.409648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:26.410785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 266/521] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:26.411662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.478031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 256/522] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:07:26 2022] POST /identity/v3/users => generated 247 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:26.478862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.518733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958f671-f123-42a4-8437-aa1b5d22c018 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.555762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.556390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.556675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.556950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.557689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:26.558010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.559401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.574026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 267/523] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:26.574811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.599158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067503-a2e9-4c8b-9612-7f5dd46d6d98 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.614018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.641592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.642499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.642499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.642752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.643486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:26.643819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.645239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.646956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.664687 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 257/524] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:26.665789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.698675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6661e-c92e-4ef2-a6aa-8c326c7a6351 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.742713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.743473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.744508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.744792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.776659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24424c6-b7c1-412a-98bc-c49a79dfcebc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.816872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.817786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.818093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.818400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.818820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:26.819550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.819816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:26.820260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:26.822155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:26.845157 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 258/525] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:26.846298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.854829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.903303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:26.927417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 268/526] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:26.928547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:26.958416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112442bd-6392-4490-952d-432c4272fd76 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:26.996512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:26.997175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:26.997458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:26.997732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.004695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.006179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:27.018204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:27.019093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 259/527] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:27.024660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.038748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f68137-2501-4997-821d-bed3fc73ff24 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.074530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.075288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.075570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.075845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.079976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.130403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321faa8-c205-40d6-977b-64ddaee5435f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.144972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.146720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.148181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.170467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.171911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.172334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.172747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.173638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=5dda95d65ff54916b0565d412c78e75f) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:27.174105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.175690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.182634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 260/528] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/role_assignments?user.id=5dda95d65ff54916b0565d412c78e75f&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:27.183367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.273519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 269/529] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:27.274720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.299652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bad7ff-81dd-46d0-8d9f-3ad2ca929b3a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.314186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.337571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.338215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.338501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.338775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.350645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.352122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:27.367858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:27.369033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:27.369904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:27.369904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:27.369904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:27.374144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 261/530] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:27.374382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.396962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4fc2-9c38-4673-9d99-f68e17af078d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.404492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.444611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.445297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.445579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.445852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.446550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:27.446858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.448721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.473433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 270/531] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:27.474912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.487014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c3229-bf05-477d-a452-79cc1c55049c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.525635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.526624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.527361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.527871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.539892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.541681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:27.557506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:27.558446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:27.559284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:27.559284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:27.559284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:27.566372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 262/532] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:27.567500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.589471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.597555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137cba7-88d9-469a-b378-9ab60d29ecde None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.634453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.635068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.635348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.635667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.636320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:27.636622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.638136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.666475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 271/533] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:27.667556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.680043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.706235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c57841-0580-4623-8456-bfaa632254b8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.751216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.751830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.752114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.752388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.753159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=8279c74cfda94068b5b392123282e556) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:27.753469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.755381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.762301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 263/534] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/role_assignments?user.id=8279c74cfda94068b5b392123282e556&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:27.762973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.804306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0f66a-24b7-4a65-bfad-0b9ad0f17692 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:27.870960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:27.871663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:27.871955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:27.872238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:27.872902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:27.873246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:27.875172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:27.889221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 272/535] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:27.895120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:27.921955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:27.983796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335edd31-b77d-4182-8944-e15f1a0014e4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.003842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdcbea-8189-420f-a42d-3ac8edc1b9b3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.040835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.041486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.041774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.042050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.049934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:28.051394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:28.063260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.063947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.064227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.064507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:28.076168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:28.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:28.077948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 273/536] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:28.082185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:28.083363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:28.083759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:28.106794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:28.109573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:28.111684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:28.114679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 264/537] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:28.119396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:28.130342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:28.165390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:28.199628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b542090-5937-4108-9af5-eae89ce0eb67 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.268246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.268867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.269168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.269438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.270130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:28.270434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:28.271867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:28.300370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 274/538] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:28.304264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963c358-ef79-40bd-ad72-55ac4061b847 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.317478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:28.362960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.363580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.363858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.364133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.364830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:28.365164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:28.366594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:28.380747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 265/539] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:28.386442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:28.474064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a27f2-2f33-4e33-a874-799625169ab7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.505355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:28.543947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.544527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.544810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.545218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.546001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=64ff5fa2e326485bb2f915fa4f8870fa) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:28.547023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:28.549169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31cd2b-3b1e-4ded-949f-57bc9d6592d4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:28.557353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:28.561563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 275/540] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:28 2022] GET /identity/v3/role_assignments?user.id=64ff5fa2e326485bb2f915fa4f8870fa&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:28.605135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876697-2ca8-4849-bef2-863140c1d51a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.658977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.659608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.659889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.660164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.660905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=ffc4548fbe5b418ea9787d78e40838a8) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:28.661239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:28.662774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad607b7-772b-424b-8f43-0e8099de3f26 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:28.670713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929fa6-d6c5-4316-9531-e4e45335a8a6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.672013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929fa6-d6c5-4316-9531-e4e45335a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.672507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929fa6-d6c5-4316-9531-e4e45335a8a6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.674427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 276/541] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:28.674541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 266/542] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:28 2022] GET /identity/v3/role_assignments?user.id=ffc4548fbe5b418ea9787d78e40838a8&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:28.680197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.680838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.681374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:28.710266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:28.711878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:28.807873 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 267/543] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:28.832195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929fa6-d6c5-4316-9531-e4e45335a8a6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:28.959258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929fa6-d6c5-4316-9531-e4e45335a8a6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:28.995743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:28.996383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:28.996664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:28.996937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.008570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:29.014049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:29.015634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:29.016594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 277/544] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:29.026321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:29.099142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:29.158565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3652b7-87c5-4870-ba33-b3ebd28a6bda None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:29.208093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:29.208711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:29.209011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:29.209293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.209974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:29.210278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:29.211689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:29.220754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 268/545] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:29.226462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:29.304387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:29.356038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7672feb-d87f-4b44-be32-a43b39f159ff None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:29.400396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:29.401125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:29.401413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:29.401740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.413298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:29.414754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:29.429759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:29.431837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:29.433904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:29.433904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:29.433904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:29.433904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 278/546] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:29.440468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:29.570448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784057a-2135-42fe-aadc-8e28c256563a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:29.591036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:29.614948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:29.615954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:29.616427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:29.616710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.617401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:29.617710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:29.619262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07d50-4d98-48fd-b381-d6f69135e91c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:29.635841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 269/547] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:29.642556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:29.689805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175342-d7be-433e-ab2a-f886b9500928 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:29.690238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175342-d7be-433e-ab2a-f886b9500928 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:29.690852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175342-d7be-433e-ab2a-f886b9500928 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.693985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 270/548] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:29.701820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:29.704468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:29.704755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.735943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:29.737158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:29.841564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846238-4472-44ef-9ad7-f64f80df2a00 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:29.877466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:29.883523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 271/549] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:29.908300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:29.912852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:29.913237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:29.913523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:29.913798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:29.926755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:29.926755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:29.939773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:29.940946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:29.941875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 279/550] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:29.946671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.088575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b63ed-14b6-460f-bbb1-2bccbc042517 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.101532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.126795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e68fdc-f316-4184-88bf-1f40ae37b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.127434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e68fdc-f316-4184-88bf-1f40ae37b80e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.138764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e68fdc-f316-4184-88bf-1f40ae37b80e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.138764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e68fdc-f316-4184-88bf-1f40ae37b80e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.139114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e68fdc-f316-4184-88bf-1f40ae37b80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-31e68fdc-f316-4184-88bf-1f40ae37b80e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:30.145172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:30.146243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:30.147051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 272/551] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:30.153047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285796d9-8775-4a9a-b61b-9b2a64add633 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.153047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285796d9-8775-4a9a-b61b-9b2a64add633 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.153047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285796d9-8775-4a9a-b61b-9b2a64add633 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.153047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 273/552] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:30.158836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285796d9-8775-4a9a-b61b-9b2a64add633 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.159535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab66bf9-19d0-49aa-b788-c525182e1807 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.200732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.201472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.201880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.203596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.204450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:30.204825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:30.206335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d11eb-c824-4a3b-be35-f6dac0193f92 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:30.231502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 280/553] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 10 13:07:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:30.233581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.234173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.234659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.257547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:30.259239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:30.276810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285796d9-8775-4a9a-b61b-9b2a64add633 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.285953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.324794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.325683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.326042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.326382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.328110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:30.329122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:30.330892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:30.337140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 281/554] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:30.338229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.349993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 274/555] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:30.362439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.406838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.480603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c94b5-830e-4f88-8567-1a581ea57f42 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.485107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50381e43-7c4e-49fc-9a8b-189a6a501f79 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.517380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.518103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.518385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.518660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.529921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:30.532096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:30.533072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.534211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.535084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.535558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:30.547044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:30.548002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:30.548936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:30.548936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:30.548936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:30.548936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 275/556] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:30.554556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.573192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.617384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:30.619210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:30.652334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.673541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546011e-313c-4cbe-aaf9-2a2325a6a4d1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.710753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.711416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.711702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.711978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.723431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:30.728527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 282/557] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:30.728661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:30.730030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:30.730854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:30.733156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.734727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 276/558] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:30.735599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.798901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.841278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8095394-a3f7-4a65-8bef-8ebcd57c0cb9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.860393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16a63e-a0fa-486a-9bbc-f68593293224 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:30.880221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.881198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.881713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.882194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.883062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:30.883561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:30.885227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:30.896917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:30.897586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:30.897872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:30.898145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:30.904488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 283/559] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:30.905771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:30.910705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:30.912351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:30.915064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:30.929045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:30.930382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:30.931814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:30.931814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:30.931814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:30.931814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 277/560] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:30.931814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.017053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2de0d-5bb3-41c7-bde0-6467d328ab3e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.036681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ff28c-80ec-4951-a87a-d5c6fb819377 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.037631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.054425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.055036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.055318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.055591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.056255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:31.056557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.057951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.068316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 284/561] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:31.069946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.076495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.077164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.077475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.077744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.086066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.087505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7ac3fec8-c043-4657-afd9-d30deac62150 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:31.100087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:31.101667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 278/562] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:07:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:31.102623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5402841-1294-476a-a449-a1b8bd09c328 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.103267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5402841-1294-476a-a449-a1b8bd09c328 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.103267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5402841-1294-476a-a449-a1b8bd09c328 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.108257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 279/563] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:31.108257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5402841-1294-476a-a449-a1b8bd09c328 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.167539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.179516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad33e02-a406-464e-9ef0-9af93c34249d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.215324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5402841-1294-476a-a449-a1b8bd09c328 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.215517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.216050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.216346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.216601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.217273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:31.217580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.219717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.232094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 285/564] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:31.233008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.256962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.257611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.257903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.258168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.269369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.270816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.277740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-918a803a-2813-4ff9-9f51-0c8f3f403681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:31.285257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:31.286225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:31.287103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:31.287103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:31.287103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:31.287103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 280/565] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:31.287560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a593ba-0e73-4cc0-bb24-120d82715365 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.287823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a593ba-0e73-4cc0-bb24-120d82715365 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.288124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a593ba-0e73-4cc0-bb24-120d82715365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.305972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-57a593ba-0e73-4cc0-bb24-120d82715365 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:31.307158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-57a593ba-0e73-4cc0-bb24-120d82715365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:31.336928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c281de-a1d2-4040-ba0c-6d5a87d0cf0d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.369923 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.370522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.370799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.371069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.371787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:31.372091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.373700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 281/566] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:31.374750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.375716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4df0-10f0-4fe0-a87a-e2b3966000d2 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.376009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4df0-10f0-4fe0-a87a-e2b3966000d2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.376289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4df0-10f0-4fe0-a87a-e2b3966000d2 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.377554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 282/567] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:31.378250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b4df0-10f0-4fe0-a87a-e2b3966000d2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.397569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 286/568] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:31.398673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.442942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.498971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.502317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7775-fb04-480d-a76b-34343d2d2e84 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.503588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b4df0-10f0-4fe0-a87a-e2b3966000d2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.536468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.537099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.537437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.537710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.538345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:31.538643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.538823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.539446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.539723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.539992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.540089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae3011-79cc-4e1f-b5da-159f05d1f416 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.546932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.548487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.551610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 287/569] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:31.552746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.553060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.553338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:31.559862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:31.560926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:31.562473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 283/570] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:31.563485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.575029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:31.576203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:31.644176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 288/571] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:31.645030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.671622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.675351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c3cfd-0022-43e7-b16f-c34aafd9e625 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.710035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.710591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.710875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.711149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.724394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:31.729151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:31.730102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:31.730938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 284/572] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:31.736570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.738299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.747491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b65251-0327-454c-a604-f8464e8ace85 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.784476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.785138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.785417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.785684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.795576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.797029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:31.811183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:31.812284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:31.813180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:31.813180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:31.813180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:31.817128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 289/573] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:31.817726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.836057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf2484-b193-47aa-b267-65f36de67b17 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.871480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.872105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.872391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.872661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.880258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.881732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:31.894003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:31.894948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 285/574] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:31.895898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.914538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.922367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b0096-5997-43a8-8b5e-3965eba2b1f7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:31.957527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:31.958124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:31.958401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:31.958664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:31.959344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:31.959644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:31.961150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:31.973230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 290/575] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:31.974004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:31.976632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:31.997143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d0389-8094-4ab3-a8ba-8f4cbea63030 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.031655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.032569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.033015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.033435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.048550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:32.053832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:32.054774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:32.055612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 286/576] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:32.056325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.081265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd11f-6543-45ac-8d0e-6abb398e5943 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.114996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.115594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.115873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.116147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.116800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.117132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.118480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.126568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 291/577] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.127436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.136008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.158291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80302d2c-3a68-45f5-a58e-31a4f725aec1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.190281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.194914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.195526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.195805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.196080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.196723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.197045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.198417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.212379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 287/578] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.212953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.230314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc2cad-2e38-4557-a08b-6f03ac1bdb70 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.264434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.265643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.265923 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.266189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.266904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.267203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.268658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.286115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 292/579] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.286896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.311456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebea95-8fdb-43e7-b2a3-e137025f684f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.345532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.346136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.346416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.346692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.347403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=1bb11e8a7a124630bf9b0e611324fabf) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.347715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.349160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.354541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 288/580] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/role_assignments?user.id=1bb11e8a7a124630bf9b0e611324fabf&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.355291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.357785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.407741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835015-d389-4946-92e3-323ba0a5331f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.436287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.449324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.450242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.450525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.450841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.451479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.451788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.453815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.463133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 293/581] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.463746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.480418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0dc4a-1604-4cdf-aed5-6da9f6f55c99 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.522468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.523101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.523386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.523662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.534980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.536434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:32.551255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:32.552214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:32.553971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:32.553971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:32.553971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:32.556978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 289/582] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:32.557822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.580442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb3fd-2caa-4ebe-b206-4c84fe24e57d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.616477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.616845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.617130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.617408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.617874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.624570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.626142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:32.637606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:32.638534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:32.639415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 294/583] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:32.640445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.674332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b471a-aaf5-44d1-9e95-f8fead08749d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.692429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.712083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.712721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.713241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.713471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.727252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.729226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:32.747413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:32.748393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:32.750044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:32.750044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:32.750044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:32.750217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9336ed6-bbae-498a-83f3-145b68f2b3f1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.753897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 290/584] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:32.754544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.789150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.790104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.790410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.790683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.803106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.804572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:32.821735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:32.822734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:32.823627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:32.823627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:32.823627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:32.823627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 295/585] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:32.823627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.870366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ed819-6e57-4e24-bd9b-fda9227ec42a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.879762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.908228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.908833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.909142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.909415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.910071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.910363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.911790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.926035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 291/586] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.926459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:32.926973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:32.935727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729f9f1-d09f-4e1d-abc4-90ddcc90d76c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:32.977009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:32.977560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:32.977838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:32.978108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:32.978800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:32.979103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:32.980505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:32.993746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 296/587] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:32.994538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.042500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d315f6-7ff3-45ac-80df-5837cbf59337 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.076827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.077998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.078432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.078700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.079399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:33.079629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.081019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.092739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 292/588] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:33.093750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.104582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89d3d4-9b35-44c0-8319-cdee10351336 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.124548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.140786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.141508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.141787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.142056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.142697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:33.142998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.144384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.161158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 297/589] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:33.161158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.176098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.209497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57369118-af5c-4f1c-82c1-d526c8f81c6c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.252402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.252963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.253521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.253799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.256805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4d84f-a049-452d-91d4-c3ea8798b624 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.263154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.265671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:33.280170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:33.282006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 293/590] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:33.282924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.295799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.297242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.297637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.297776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/5dda95d65ff54916b0565d412c78e75f/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.313873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=5dda95d65ff54916b0565d412c78e75f, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.315325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.369009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.383675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455afb2-e878-4d50-acd1-f58e8427b199 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.411356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 298/591] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:33 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/5dda95d65ff54916b0565d412c78e75f/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:33.411356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.435541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.436896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.437270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.437554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.445839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.447401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:33.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:33.462694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:33.463573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 294/592] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:33.469583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.503449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.554922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1050e-a51c-45ce-8019-5ada570e27bf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.597579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.598217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.598327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69e75b-d242-4c5d-ab64-f76621c9db10 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.598498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.598767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.606274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.607527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:33.620408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:33.621412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 299/593] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:33.622333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.641455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.642629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.642977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.643322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.644210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:33.644587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.646424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.669413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 295/594] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:33.670513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.674776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.745496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae64c2-e16c-4a75-b186-1348f2804baa None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.774324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.783924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.784335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd40a4-89f4-4063-b7f3-bfc73d376be4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.786549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.786549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.786549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.786549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:33.787212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.788889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.803220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 300/595] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:33.804377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.827810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.828637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.828910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.829227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.830712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:33.831088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.833221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.852475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 296/596] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:33.853434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:33.915044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c983eb-a7a4-4d94-80f3-0d087515f152 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.923113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:33.951937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:33.953413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:33.953838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:33.954251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:33.965296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b29c0-0e62-4581-ada4-ed9e2517f4c7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:33.967930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:33.970550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:33.992480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:33.993487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:33.994371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:33.994371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:33.994371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:33.995094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 301/597] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:33.995421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.014507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.016085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.016600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.017194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/8279c74cfda94068b5b392123282e556/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.036323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=8279c74cfda94068b5b392123282e556, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.037895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.044516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.105236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854ef3d-8b1e-4ad7-8e8f-394974c88fb6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.120063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 297/598] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:33 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/8279c74cfda94068b5b392123282e556/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:34.125690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.164906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.165746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.166162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.166715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.181212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.183251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:34.202494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:34.203634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:34.204506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:34.204506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:34.204506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:34.204506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 302/599] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:34.209348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.333405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60cc2-bb88-469e-8ebf-97ad4fe0831f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.349269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.372537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.373190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.373472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.373751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.375029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:34.375332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.376827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.390320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 298/600] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:34.392601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.396024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.404075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a685ae7-e62e-494f-92c9-f24236fe3992 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.440395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.441026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.441344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.441613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.442259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:34.442551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.443922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af659c-4101-462f-8845-88f6d5fed703 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.457931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 303/601] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:34.459185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed9323-91aa-421d-90f9-5b5893220bcc None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.459472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed9323-91aa-421d-90f9-5b5893220bcc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.459746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed9323-91aa-421d-90f9-5b5893220bcc None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.461215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 304/602] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:34.466730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.467013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.467289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.486833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:34.488020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:34.524535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da060b98-06af-494b-8cf6-3dda63296c1e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.558826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 305/603] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:34.559747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.567397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.568459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.568744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.569040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.578344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.580110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:34.599457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:34.600352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:34.601272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 299/604] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:34.601761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.641008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.657041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.679184 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491f458-0a60-49f6-9c9b-2e169d67e54e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.720954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.721682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.721945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.722280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.730291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.732603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:34.748703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:34.750885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:34.751977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:34.752485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcad36d-fe6b-4507-a170-6deba709b6f1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.755046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 306/605] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:34.756003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.793860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.794642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.795045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.795523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.808193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:34.816483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:34.817694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:34.818726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:34.825160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 300/606] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:34.825160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.894545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0179d-9524-42b3-82b6-764ead8743d8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.923881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.933027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9433ec1-fb01-47b0-9845-01581ebe28f5 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:34.946954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.948207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.948634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.949194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.950141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:34.950604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.952939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:34.965797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:34.971304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:34.971756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 307/607] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:34.972793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:34.977105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:34.977332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:34.977676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:34.978467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:34.984196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:34.984196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:35.000279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 301/608] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:35.001386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:35.098755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d36b9-3939-4123-80f8-d26dee82338b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.126435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21583a-0103-4e18-a15f-0ce0ef9d9176 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.134943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:35.135815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.136202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.136571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.137417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:35.137823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:35.139284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:35.153861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 308/609] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:35.153861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:35.175118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:35.175840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.176120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.176394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/64ff5fa2e326485bb2f915fa4f8870fa/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.193742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=64ff5fa2e326485bb2f915fa4f8870fa, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:35.195259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:35.229912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:35.269613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cac30-bb85-457f-82bd-7154f5755ac1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.301488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 302/610] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:35 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/64ff5fa2e326485bb2f915fa4f8870fa/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:35.305928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:35.313022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:35.314087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:35.315624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.315624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.315624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/ffc4548fbe5b418ea9787d78e40838a8/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.343148 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=ffc4548fbe5b418ea9787d78e40838a8, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:35.347169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:35.453527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 309/611] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:35 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/ffc4548fbe5b418ea9787d78e40838a8/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:35.454857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9d8a-06e1-45f3-bad6-a14b27c3f12d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.498109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:35.498772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.499784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.500199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.512618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:35.514876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-bb269ca2-a0d5-4aee-9849-2d14e9e742a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:35.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:35.533227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:35.534123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:35.534123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:35.534123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:35.534123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 303/612] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:35.542404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:35.645200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7ced-4b93-429a-b495-334f0d39f7e7 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.645506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7ced-4b93-429a-b495-334f0d39f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.645778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7ced-4b93-429a-b495-334f0d39f7e7 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.652743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 304/613] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:35.658431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.658715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.659836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.682966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:35.684099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:35.691182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d582f-881c-4fe8-b47e-a16bb212909d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.709458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:35.736224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:35.737163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.737447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.738100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:35.739092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:35.739349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:35.740810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:35.741290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:35.756774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 310/614] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:35.763540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:35.765953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 305/615] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:35.790540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:35.930373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f949a37-80a8-4ae8-b9aa-3d72a7307627 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.968912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874489e4-94fd-47e5-a988-de9f367ee310 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:35.993039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:35.993686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:35.993966 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:35.994239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.007815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.009332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:36.029391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:36.030963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:36.031908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 311/616] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:36.037077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.050353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.051016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.051302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.051578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.078462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:36.083590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:36.084550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:36.085421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:36.086062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:36.090584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 306/617] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:36.091763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.157323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154360b-4670-4dbf-a8de-4fe28b1a9b4d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.195861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.196822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.197347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.197983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.199408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:36.200079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.201750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.206799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:36.236670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 312/618] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:36.237058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.305037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac8a8-6a5d-498d-9b5e-9fbcf4394e7e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.339851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.340469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.340754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.341059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.341734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:36.342037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.343504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.352189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 307/619] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:36.357676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.464481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4922-7a8e-47f9-929a-3634cde83b3d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.466831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:36.474157 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:36.480568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10b99b-5b3a-4ff3-9e0c-185000a6c97d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.502805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.503637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.504028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.504403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.516529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.518223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.523518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.524378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.524796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.525281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-aa046373-0037-4bc2-95c7-88a03cdfbb3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:36.535014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:36.536053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:36.536921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:36.536921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:36.536921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:36.537407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 308/620] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:36.539919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.541784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.542875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874244-8da3-4cf2-9429-ae7c79fa562c None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.543231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874244-8da3-4cf2-9429-ae7c79fa562c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.543574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874244-8da3-4cf2-9429-ae7c79fa562c None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.549234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 309/621] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:36.549234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56874244-8da3-4cf2-9429-ae7c79fa562c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f5e58c6a-3b4f-4bdf-b0c5-4e11af628db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:36.563206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:36.564140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:36.565531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:36.565531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:36.565531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:36.565531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 313/622] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:36.570214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.570501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.570779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.590205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:36.590965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:36.661210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56874244-8da3-4cf2-9429-ae7c79fa562c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.685027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 314/623] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:36.685027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.722453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.723410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.723886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.724372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.725269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:36.725765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.727307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.741534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 310/624] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:36.743340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.764353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:36.777188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:36.817551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81035f-e7bd-4478-a452-916af3367ab0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.865712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.866686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.867144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.867763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.868896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:36.869645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:36.871320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:36.878051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319968-f113-4893-b7b2-1d5943aeddf1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:36.896986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 315/625] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:36.898581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:36.923812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:36.924553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:36.924908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:36.925279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:36.939816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:36.945957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:36.946939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:36.947763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 311/626] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:36.948750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.029369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354740-f210-417f-88b6-a4e5f16c73e1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.054204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.067205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.074192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.075402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.076137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.076616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.081972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9bec-ad12-4cd0-8f62-776b0940d249 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.087344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.089056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:37.104046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:37.105032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:37.106040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:37.106581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 316/627] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:37.112221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.131871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.132553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.132840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.133144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.141441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.143025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:37.159484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:37.160455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:37.161756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:37.165542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 312/628] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:37.166503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.242455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d9090-6211-4163-965a-f6cc915821c5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.278024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e5aa4-691d-4e8a-8fe6-57829e149927 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.290605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.291826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.292313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.293014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.294365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:37.295142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.297587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.309377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 317/629] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:37.311021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.316827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.317148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.317444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.317725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.318461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:37.318773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.320216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.330079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.345413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 313/630] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:37.346386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.384553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.431878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55950e38-5526-4743-9eb7-bac5f98d9886 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.469741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.470390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.470647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.470974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.471677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:37.472204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.473441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.478858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95ba9e-8cb4-4326-b2b1-dbbca2de58ea None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.491692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 318/631] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:37.492565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.535214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.537256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.537965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.539167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.562640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.564806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:37.593658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:37.595153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:37.596591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:37.596591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:37.596591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:37.603559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 314/632] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:37.603559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.613447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7811bf5-80ea-4ae8-b22b-915cc1651af6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.628454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.651405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.658190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.658190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.658190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.675786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.729042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.730762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.746380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf308f-78b4-4bb8-93ce-3740f17176e1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:37.782944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5807e9-cf93-4f5f-8679-2f000244967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:37.783618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5807e9-cf93-4f5f-8679-2f000244967e None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.783896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5807e9-cf93-4f5f-8679-2f000244967e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.784178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5807e9-cf93-4f5f-8679-2f000244967e None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/1bb11e8a7a124630bf9b0e611324fabf/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.799448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5807e9-cf93-4f5f-8679-2f000244967e None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=1bb11e8a7a124630bf9b0e611324fabf, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:37.801034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5807e9-cf93-4f5f-8679-2f000244967e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:37.835036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 319/633] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:37.835872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.850524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:37.871910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 315/634] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:37 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/1bb11e8a7a124630bf9b0e611324fabf/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:37.873179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cd753-9857-4454-9cc3-a013d44e8b3a None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.873465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cd753-9857-4454-9cc3-a013d44e8b3a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.873737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cd753-9857-4454-9cc3-a013d44e8b3a None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.880158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 316/635] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:37.880158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2fab5-bdbc-4c99-af6a-e05994fa0275 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:37.880158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2fab5-bdbc-4c99-af6a-e05994fa0275 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:37.880158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2fab5-bdbc-4c99-af6a-e05994fa0275 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:37.880589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 317/636] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:37.881440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2fab5-bdbc-4c99-af6a-e05994fa0275 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:37.963421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28778f8c-6596-4d03-b7db-d6b95acd2695 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.001448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.004509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.005179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.005467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.005798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.006296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2fab5-bdbc-4c99-af6a-e05994fa0275 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.006522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:38.006829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.008515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf25d89-0242-4442-a8a4-a36bd6a632e8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.027618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 320/637] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:38.028667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.028956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.029263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.049735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.050438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.051072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.051459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.052841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:38.054218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:38.062113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.063578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-dd70fbc4-4205-4187-a659-c7422a6705ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:38.078549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:38.079683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:38.080557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:38.080557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:38.080557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:38.082901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 318/638] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:38.083842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.084130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.084405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.103274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:38.104401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:38.135062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 321/639] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:38.135912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.146277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.194667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 319/640] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:38.197893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.258626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576f4bd-4ed5-4991-aae9-c4346aee9b1d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.287436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.295583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.297052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.297334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.297600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.304793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.306257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ceba5-d20f-4253-b148-383d8fcb4c10 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.306387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:38.318868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:38.319831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 322/641] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:38.320794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.350538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.351155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.351443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.351714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.352364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:38.352666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.354107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.370467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 320/642] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:38.370467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.394809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.434387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8feb43-2a42-4854-a53c-cabb3c1cf389 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.468771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.469427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.469703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.469969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.481602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:38.482654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49f2ae-6d31-44ef-a731-e1206adddd81 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:38.487738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:38.488711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 323/643] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:38.494042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.525061 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.525796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.526078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.526440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.534731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.538591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:38.544019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:38.544939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:38.546434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 321/644] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:38.547233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.597266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bda0a-a24d-4dba-9e4d-01b56e69f67e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.621423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.633094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.633711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.633989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.634261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.634986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:38.635294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.636710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.660459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 324/645] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:38.661588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.666711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67e43-3705-4f63-8f3e-b603dae65752 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.702089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.702751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.703028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.703291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.710168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.711674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:38.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:38.725288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 322/646] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:38.726195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.772203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e65a8-c3ee-42e3-ba78-55a87f7f09fb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.790275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.811176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.812422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.812701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.812969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.813655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:38.813953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.815356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.825040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 325/647] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:38.825405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.837966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644116-5bca-4121-af9a-fef6e71f5db0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.871946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:38.875718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.876339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.876620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.876898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:38.877645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:38.883984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:38.883984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:38.892592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 323/648] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:38.892592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:38.940243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07baa4-73e3-42ad-ae91-dbd729c07ef4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:38.976287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:38.976641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:38.976844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:38.977146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.007358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708185-c9ee-46e1-a417-fbb137de058b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.038422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.051532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.052240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.052518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.055483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.055483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:39.055483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.055483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.059214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.060641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.074673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 324/649] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:39.074673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.111624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.172941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 326/650] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:39.172941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.176689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d6850-2a04-442a-b0e7-5008528e5edc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.211271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.212185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.212648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.213156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.223534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.224987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:39.240415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:39.241440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:39.242424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:39.242424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:39.242424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:39.242424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 325/651] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:39.242424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.280083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a97d-dda8-4473-89fa-d87f8a8bd7c6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.281274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.318525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.319158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.319437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.319707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.331145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.332624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.335428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:39.348552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:39.349546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:39.350426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:39.350426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:39.350426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:39.351075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d1f0-e8a2-4404-adcc-366d74d0efcf None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.354515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 327/652] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:39.355136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.385282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.385886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.386170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.386444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.387165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=5dda95d65ff54916b0565d412c78e75f) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:39.387470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.388905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.405028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 326/653] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/role_assignments?user.id=5dda95d65ff54916b0565d412c78e75f&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:39.405028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.478175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4a5d-252f-4a75-a5d4-35b0eeeb1310 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.523652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.525641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.526398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.526781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.527466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.528454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6a5e1-9af8-4fdc-b630-b31743535658 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.540405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.542083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:39.558528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:39.559710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:39.560776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:39.560776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:39.560776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:39.562966 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 328/654] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:39.563883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.569261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.569932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.570214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.570487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.571197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:39.571505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.572905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.592023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 327/655] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:39.593623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.630807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.705958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37cf53-08f2-4352-b152-46a4f07b0a19 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.722383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34763-6410-438f-9869-d70386f62934 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.751850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.752383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.752661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.752917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.753610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:39.753910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.755263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.770150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.771052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 329/656] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:39.771184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.771603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.772006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.772165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.773147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:39.773611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.775500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.792633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 328/657] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:39.794069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.832935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.897524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a0afa-0137-414b-9adc-7b98f4804e66 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.908518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:39.935745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.936193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.936632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.936972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:39.944575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:39.946855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:39.951710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630271f-a4b1-4c30-ab54-553871208f8d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:39.960234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:39.961437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:07:39.962554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:39.971083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 330/658] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:39.971083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:39.997323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:39.997963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:39.998244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:39.998518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.007191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.008558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-4a50b1a1-e1ad-4b64-8027-9dbb7bd2518c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:40.019867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:40.020963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:40.022237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 329/659] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:40.026655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9926f5f-5ae5-4ece-813d-398530c1daf2 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.027199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9926f5f-5ae5-4ece-813d-398530c1daf2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.027483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9926f5f-5ae5-4ece-813d-398530c1daf2 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.033126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 330/660] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:40.033930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9926f5f-5ae5-4ece-813d-398530c1daf2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.079779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1699fe0-438e-442c-969e-3a82ab0c5790 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.118649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.119295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.119572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.119843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.121767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.129136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.130517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:40.142365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:40.148549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 331/661] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:40.149492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.153660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9926f5f-5ae5-4ece-813d-398530c1daf2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.175026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.190629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.191292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.191571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.191843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.192570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:40.192875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.194339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a6d8-10de-4013-8d32-25d6ee122eb3 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.213680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 331/662] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:40.215199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.215635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.216059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.235032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:40.236213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:40.260609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b615652-a5ec-4af5-a262-9a1a765a32ee None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.296912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.297979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.298561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.299023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.299935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:40.300528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.302288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.305389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 332/663] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:40.306171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.327356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 332/664] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:40.328265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.379995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.415183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.422382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff771c40-ca2e-423d-9933-d06587e54584 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.429900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8842-5574-4d5a-b9cf-92f5b3ffaf15 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.457737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.458341 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.458616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.459050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.459734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:40.460036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.461620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.467340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.469214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.469600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.469975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.473688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 333/665] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:40.474282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.548532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.550012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.578027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6c2bf-2910-46dd-809e-70587a22ea64 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.604356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.613994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.614611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.614893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.615168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.638570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.660901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 333/666] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:40.661796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.691833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.693363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.769286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327a64-1ad1-4e63-8697-244a42b438e8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.798939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 334/667] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:40.799709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.810014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.810649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.810929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.811205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.822521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:40.828109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:40.829387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:40.830432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:40.831230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.832574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 334/668] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:40.838274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.888252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:40.922959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05acf0e-916b-4591-8761-2abb99112095 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.943571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5033-5116-49fa-8528-eef7e0ee01f1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:40.958046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.958654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.958938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.959211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:40.959939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=8279c74cfda94068b5b392123282e556) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:40.960251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:40.961706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:40.967347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 335/669] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/role_assignments?user.id=8279c74cfda94068b5b392123282e556&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:40.973216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:40.993871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:40.994790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:40.995139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:40.995587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.007780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.009395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:41.025847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:41.026915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:41.028107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:41.028107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:41.028107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:41.028107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 335/670] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:41.028701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.095019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.105461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4f7e-5b67-4d0c-ad08-a10213aa8d8d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.153519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.154629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.155956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.156558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.163545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.168575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0ae5-9a41-4678-924b-3991fd51d138 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.169730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.171910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:41.188351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:41.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:41.190407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:41.190407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:41.190407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:41.193347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 336/671] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:41.194208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.211379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.212006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.212288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.212565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.213276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:41.213586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.214950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.224741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 336/672] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:41.225621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.322076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c58c7-68e2-4950-85b4-f429eb036c23 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.359266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd87c5-9cd6-45fb-8fca-3aa63a2b21f8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.369891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.370618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.370974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.371324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.372368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:41.372746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.374999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.388376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 337/673] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:41.394252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.397684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.399151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.403267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.403267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.403267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.403267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:41.403267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.403773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.417638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 337/674] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:41.418768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.454391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.530091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca69439-1170-46c6-a5b0-81d56e3b69c0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.559139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd68b-d026-4662-ba06-42fb6626d46c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.576496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.578283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.579731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.580477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.594585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.596880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.603109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.603748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.604028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.604304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.613166 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:41.613465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:41.614359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:41.615240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:41.615240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:41.615240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:41.615441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.615521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 338/675] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:41.616148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-866d4c25-9c22-4867-ad30-5cd300984175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:41.633360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:41.634330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:41.635209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 338/676] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:41.636325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b151d-83d0-4454-8dcd-128efeb3ba66 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.636325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b151d-83d0-4454-8dcd-128efeb3ba66 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.636475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b151d-83d0-4454-8dcd-128efeb3ba66 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.641208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 339/677] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:41.641208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b151d-83d0-4454-8dcd-128efeb3ba66 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.684952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.731224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59007a1e-20de-4eef-945b-03b18e9a6ef6 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.737435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.763271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b151d-83d0-4454-8dcd-128efeb3ba66 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.766491 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.767128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.767413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.767689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.774713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.776155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:41.788446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:41.789567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:41.790651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:41.793329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 339/678] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:41.794391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.806868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.807606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.808019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.808365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.809144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:41.809548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.811007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8286ca-a03c-4aca-a702-d9790af40a99 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.828221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 340/679] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:41.828221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.828221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.828221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.843909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:41.845224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:41.912778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3131e-345e-4e24-8d41-cccf172e1f23 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:41.921057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 341/680] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:41.921057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:41.934150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:41.958001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:41.958840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:41.959268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:41.959686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:41.960520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:41.961017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:41.962673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:41.975426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 340/681] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:41.976388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.007315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.038374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a041b57-7798-4079-a58b-8a5d95a9f695 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.075237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.075999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.076281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.076870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.078365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:42.078678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.080107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.082664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c2312-b588-496b-b410-bd436f1a0e3e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.096957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 342/682] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:42.096957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.127573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.128215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.128496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.128769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.138260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.140583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:42.156813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:42.157791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:42.158727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:42.159278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 341/683] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:42.159832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.197571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a182-647e-4785-bcf1-56af26f4e935 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.199195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.227799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.236893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.237587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.237855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.238125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.250208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:42.255291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:42.256225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 343/684] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:42.257488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.274426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af945-7344-4545-9bb8-3411203660d9 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.309698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.310305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.310593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.310867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.311680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=64ff5fa2e326485bb2f915fa4f8870fa) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:42.311990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.313465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.329096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 342/685] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/role_assignments?user.id=64ff5fa2e326485bb2f915fa4f8870fa&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:42.329096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.382092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef66e4f-d52e-4cd8-90ef-664cc90b05b4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.424395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.425279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.425919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.426274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.427122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=ffc4548fbe5b418ea9787d78e40838a8) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:42.427540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.429117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.442405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 344/686] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/role_assignments?user.id=ffc4548fbe5b418ea9787d78e40838a8&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:42.443409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.461447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.467057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486484d3-39ef-4d6b-acb5-b55260741ab6 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.513696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.515337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.515652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.515996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.516758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:42.517091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.518528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.540359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 343/687] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:42.550989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.573471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.640847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfab69-1152-4fcf-82bb-d6df0731768d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.682416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.683343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.683790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.684066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.684816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:42.685311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.687198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.690719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5937-6693-4413-ade1-8b6c3aaa8c99 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.697790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 345/688] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:42.705891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.735949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.736869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.737878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.738326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.775803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.825840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.837288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd680a-068a-4c7c-91a5-511821cb5bcc None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:42.856036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:42.876740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:42.877681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:42.878179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:42.878616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:42.892048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:42.893837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:42.910214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:42.911224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:42.912112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:42.912112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:42.912112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:42.914314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 346/689] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:42.920588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:42.950034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 344/690] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:42.961994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.057372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.074309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979c00f-1f31-491e-adf4-82632ba78b59 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.131847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.132798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.134485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.134967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.136817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:43.137332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.140083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.142637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c115f-3f96-476e-b5f8-d4b5ce4e9f65 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.166272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 347/691] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:43.174536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.210570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.211209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.211490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.211766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.223769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.225237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.238860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b90b909f-641c-4118-8f2e-9baedfb6063b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:43.250594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:43.251656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:43.252772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:43.252772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:43.252772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:43.252772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 345/692] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:43.258453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1893e6-f175-4ef1-bfa2-a1282e026a26 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.258755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1893e6-f175-4ef1-bfa2-a1282e026a26 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.259021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1893e6-f175-4ef1-bfa2-a1282e026a26 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.260720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 346/693] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:43.261469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1893e6-f175-4ef1-bfa2-a1282e026a26 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.325540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cc8c1-da60-48bd-af38-eff1fe18082c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.365475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.366122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.366873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.367933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1893e6-f175-4ef1-bfa2-a1282e026a26 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.376254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.378273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.381373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6e363f8f-8f9d-4ecb-8836-ffe48513b86d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:43.393945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:43.395919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:43.396869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 348/694] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:43.397503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.397503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.398062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.416755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.417435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.417718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.417993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.418640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:43.418942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.419700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:43.421019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:43.422031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.438857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 347/695] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:43.439709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.502060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 349/696] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:43.503040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.523604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.548044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05470e9c-6a9d-4036-9f4a-3dbdba54b1b1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.588967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.589428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.589718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.589992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.590736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:43.591042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.592482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.611161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 348/697] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:43.615759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.622865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a00592-c81d-4fa6-8001-1a6479ba52a1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.641181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.670012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.671024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.671707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.673111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.683057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.685189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:43.700301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:43.701353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:43.702809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:43.705068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 350/698] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:43.706088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.741036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69122683-1533-4c92-9e0a-fe566607df7d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.797019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.797019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.797019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.797019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.797433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.805519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.807742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.818517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66c87-c387-4a07-9529-5529bbb1635b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:43.828108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:43.829361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:43.830637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:43.830637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:43.830637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:43.830637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 349/699] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:43.833992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:43.863391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.864075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.864359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.864631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.931078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:43.950398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.952383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:43.959789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb69249-121c-438e-b640-cde068262d48 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:43.995817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:43.996428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:43.996707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:43.996977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:43.997696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:43.998000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:43.999996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.013521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 350/700] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:44.014464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.059149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:44.090579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 351/701] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:44.091799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.127056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab828-f5d4-45e5-b1d7-bf1c1f8c4fd2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.163608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:44.164471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.164827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.165247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.165978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:44.166410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:44.167915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.175309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:44.185213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 351/702] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:44.186982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.225859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1de052-d0e5-4c59-be65-3257bba7f9d1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.271907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:44.272753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.273569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.273992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.275127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=1bb11e8a7a124630bf9b0e611324fabf) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:44.275580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:44.278161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.287043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 352/703] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/role_assignments?user.id=1bb11e8a7a124630bf9b0e611324fabf&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:44.288415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.298681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcc3cb-5931-42f9-954b-bd89ee978932 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.340531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:44.347009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:44.348262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.348742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.349363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.363115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:44.365077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:44.383090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:44.384069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:44.384954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:44.384954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:44.384954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:44.387882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 352/704] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:44.395426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.455317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0405b5c-e7b3-4cdb-ab32-5417ffc61dc6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.503474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:44.506354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.507732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.509011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.510030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:44.512938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.514161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:44.581091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb03752-7d85-4c6b-8b1a-d2c7fbeab717 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.612386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 353/705] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:07:44 2022] POST /identity/v3/users => generated 254 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:44.623697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:44.624605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.624888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.625210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.625928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:44.626289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:44.627679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec4631-8852-47a2-954c-c8580ca5ae9f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.643054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 353/706] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:44.657977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.770042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:44.774528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4635-fe19-44c9-bcc8-c764f7418d9e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.810277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:44.810917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.811194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.811995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.819838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:44.821573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98f2bf-24da-4366-8700-e3f49728ce19 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.822028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:44.822109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98f2bf-24da-4366-8700-e3f49728ce19 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.822109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98f2bf-24da-4366-8700-e3f49728ce19 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.823603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 354/707] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:44.831728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:44.832017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:44.832285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:44.834334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:44.835226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:44.836109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 354/708] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:44.844591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:44.856085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:44.858095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:44.961098 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 355/709] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:44.971922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f43c1b-8c47-4857-b64e-cf49ae0a4bc9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:44.981239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.019511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.020493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.020983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.021490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.022379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:45.022881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.024530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.038851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:45.042268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 355/710] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:45.048521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.071526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:45.107213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383dc01-6665-4e29-8138-fe3926f73052 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:45.146089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.147164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.147711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.148182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.148967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.150768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-92d8208f-5125-482e-94cd-9ef71804c779 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:45.163019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:45.164075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:45.164931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:45.165700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc290792-3847-43b9-ba62-1ef324273992 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:45.167950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 356/711] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Aug 10 13:07:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:45.174826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c086-0a22-4c31-9c25-25e9b5343bac None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.175459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c086-0a22-4c31-9c25-25e9b5343bac None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.175936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c086-0a22-4c31-9c25-25e9b5343bac None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.177809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 357/712] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:45.179431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7c086-0a22-4c31-9c25-25e9b5343bac None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.203484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.204107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.204392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.204667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.205435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=5dda95d65ff54916b0565d412c78e75f) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:45.206172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.207269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47f471-8cae-4988-ab63-6520ccf3da87 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.218654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 356/713] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:45 2022] GET /identity/v3/role_assignments?user.id=5dda95d65ff54916b0565d412c78e75f&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:45.218654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.218654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.218654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.236289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:45.237696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:45.317228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 357/714] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:45.339459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.356924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7c086-0a22-4c31-9c25-25e9b5343bac None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:45.362401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:45.364950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:45.439635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.440337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.440625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.440899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.441578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:45.441877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.443305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.458932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 358/715] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 10 13:07:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:45.473185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.566442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e2fc3-2543-4662-a95f-d3d7654fac9a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:45.604187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75db93b-7294-437e-8db7-0edc939769e6 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:45.604885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.605814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.606260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.606762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] PATH_INFO: `/v3/services/network` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.607529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.610346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:45.623518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:45.625871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:45.626741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:45.635378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 358/716] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:45 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:45.638299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.675860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.676769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.677252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.678241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.680264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:45.680747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.683515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.697991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 359/717] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 10 13:07:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:45.769384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:45.784000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd3a0-4e97-4920-9cd6-59fad55d75bb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:45.821434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:45.822052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:45.822330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:45.822600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:45.823232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] RBAC: Adding query filter params (name=network) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:45.823530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:45.824906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e70f52-d00a-4601-9e5a-6f5cabf7ac7c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:45.835884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 359/718] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:07:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:45.837293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:45.886480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:45.985766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d0cea-3a22-43d8-917f-d1516f646680 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:46.028595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:46.029162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:46.029441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:46.029711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:46.030447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] RBAC: Adding query filter params (type=network) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:46.030745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:46.032122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2d590-445d-4617-95a4-d3faa0c41d2d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:46.036891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 360/719] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:07:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:46.938365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b758f-01b4-4aba-be2c-1a9cb2a87ce5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:46.938853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b758f-01b4-4aba-be2c-1a9cb2a87ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:46.938853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b758f-01b4-4aba-be2c-1a9cb2a87ce5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:46.940388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 360/720] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:46.945817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a07e8-e7c6-4b16-ad06-46fe6e45f7d4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:46.946241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a07e8-e7c6-4b16-ad06-46fe6e45f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:46.946429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a07e8-e7c6-4b16-ad06-46fe6e45f7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:46.971060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-951a07e8-e7c6-4b16-ad06-46fe6e45f7d4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:46.972237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-951a07e8-e7c6-4b16-ad06-46fe6e45f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:47.067507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 361/721] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:47.102547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b758f-01b4-4aba-be2c-1a9cb2a87ce5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:47.116685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152a7b5-38d4-47d8-9953-4d19ad9b61c3 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.117310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152a7b5-38d4-47d8-9953-4d19ad9b61c3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.117943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152a7b5-38d4-47d8-9953-4d19ad9b61c3 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.119854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 362/722] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:47.125599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.126255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.126732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.166014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:47.167667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:47.207845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:47.219638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b758f-01b4-4aba-be2c-1a9cb2a87ce5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:47.259553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:47.260338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.260621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.260892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.270603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:47.278292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:47.284449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:47.295308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:47.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 361/723] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:47.299466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:47.323321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 363/724] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:07:47.363911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:47.581032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ea395-6f1e-4758-a2a1-e83c9f371964 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:47.595673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94905b-cc4e-4201-82b2-c301814ec1ba None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:47.632901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:47.633542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.633821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.634093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.634953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:47.635262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:47.636683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:47.638245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:47.638885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.639163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.640059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.646792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 362/725] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:47.660539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:47.661742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:47.666421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:47.667354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:47.668198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 364/726] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:47.677963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:47.719539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:47.735260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:47.784465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc13bc-7d20-4de7-842f-1c133923855f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:47.824552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:47.825563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.826381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.827028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.839668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:47.841714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:47.857615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:47.859468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:47.860360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:47.860360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:47.860360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:47.862314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 363/727] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:47.869039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:47.910310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cda38-8542-484a-9474-f558da83ef90 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:47.949585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:47.950858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.951308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.951722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.952665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:47.953144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:47.954634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad13a8b-8dd4-45bf-97b0-82f0cb82c7b3 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:47.963143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 365/728] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:47.969518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dc609-8da5-40e8-9de3-eedf41d23770 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:47.970033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dc609-8da5-40e8-9de3-eedf41d23770 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:47.970459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dc609-8da5-40e8-9de3-eedf41d23770 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:47.972109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 366/729] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:47.972961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dc609-8da5-40e8-9de3-eedf41d23770 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.009816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d383461-0fdf-47fe-9bdc-7994c5294cd6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.024866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.047419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.047975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.048308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.048586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.049750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:48.049750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.051080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd92612-1f08-4ef2-a19e-95fc113a85b1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.058191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.065022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 364/730] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:48.066542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.066959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.067407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.086239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dc609-8da5-40e8-9de3-eedf41d23770 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.087065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:48.092716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:48.121889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.122622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.122967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.123394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.134238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.135748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:48.152740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:48.153927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:48.155171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:48.155171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:48.155171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:48.160279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 367/731] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:48.161249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.173195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 365/732] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:48.174374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.263017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3718a2f-a1e9-42df-89bb-115ec4fce055 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.292517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.299499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.300373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.300749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.301149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.303341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb55fbd-6dac-4263-9ca3-05b74f925d77 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.309652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.311609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.314114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:48.322986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:48.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:48.324887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:48.329571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 368/733] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:48.330531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.346102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.346102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.346102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.346102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.346534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:48.346534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.347854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.365798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 366/734] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:48.365798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.445499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c93d4-ae0d-444c-a69a-f7a727e2efe7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.467795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaa2ff-2ce6-403f-938e-ae9842392f29 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.482175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114f105-67b8-4677-9e85-d232fa469844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.482812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f105-67b8-4677-9e85-d232fa469844 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.483127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f105-67b8-4677-9e85-d232fa469844 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.483506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f105-67b8-4677-9e85-d232fa469844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.494257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f105-67b8-4677-9e85-d232fa469844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-e114f105-67b8-4677-9e85-d232fa469844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:48.500193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:48.501219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:48.502056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:48.503195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.503195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.503195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.503195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.503454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:48.503454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.503821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.506418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 369/735] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:48.507701 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6f8e-140f-48b0-b4c2-574351c30a75 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.507987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6f8e-140f-48b0-b4c2-574351c30a75 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.508257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6f8e-140f-48b0-b4c2-574351c30a75 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.513158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 370/736] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:48.513158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b6f8e-140f-48b0-b4c2-574351c30a75 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.528618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 367/737] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:48.529713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.553108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.566461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.618855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b6f8e-140f-48b0-b4c2-574351c30a75 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.633395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efb4df-27d4-44a2-9a7b-0dedb18ea20c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.658279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.658923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.659205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.659475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.667302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.668731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.678477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.679266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.679689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ec064462-e818-4e9f-b047-d6e56bbd9632 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:48.680070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:48.681189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:48.682128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 371/738] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:48.682719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.682719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.682848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.682986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.683783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:48.684220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.685785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.698986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 368/739] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:48.699962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.707828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:48.709466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:48.776371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 372/740] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:48.777564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.782520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.800489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583b01-25ef-41a3-ba9c-c60179dfb67a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.829779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:48.833523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc00ae-e375-43cc-bc00-806591986bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.834133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc00ae-e375-43cc-bc00-806591986bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.834419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc00ae-e375-43cc-bc00-806591986bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.834693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc00ae-e375-43cc-bc00-806591986bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.887909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb06be1-a66e-4c55-9b45-d936f1f9f944 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:48.901731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc00ae-e375-43cc-bc00-806591986bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.903785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc00ae-e375-43cc-bc00-806591986bf5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.922219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:48.922828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:48.923109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:48.923389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:48.924113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:48.924836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:48.926289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:48.949803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 373/741] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:48.949803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:48.986637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.002071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 369/742] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:49.003304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7917c-a679-47e6-98a3-aa3c07ff9f33 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.003589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7917c-a679-47e6-98a3-aa3c07ff9f33 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.003863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7917c-a679-47e6-98a3-aa3c07ff9f33 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.005131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 370/743] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:49.005813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7917c-a679-47e6-98a3-aa3c07ff9f33 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.053613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.057781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c0f54-af67-4696-9d35-88a5eb8d8202 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.091455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.092081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.092359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.092664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.102617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.104056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.106849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7917c-a679-47e6-98a3-aa3c07ff9f33 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:49.119194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:49.120334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:49.121289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:49.121289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:49.121289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:49.121289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 374/744] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:49.121289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.144943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.145513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.145791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.146061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.146692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.148044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.221989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49a318-edd2-4663-98b9-f060617d329f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.254589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552887b-7d0c-4632-b2bc-5d54ed7e5e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.255186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552887b-7d0c-4632-b2bc-5d54ed7e5e7e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.255464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552887b-7d0c-4632-b2bc-5d54ed7e5e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.255736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552887b-7d0c-4632-b2bc-5d54ed7e5e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.266030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 371/745] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:07:49 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:49.268879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.278074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.336321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.338617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552887b-7d0c-4632-b2bc-5d54ed7e5e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.340407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552887b-7d0c-4632-b2bc-5d54ed7e5e7e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.413962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8b0e-5601-41dd-9a82-b251e54f4a57 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.456983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.457227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 375/746] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:07:49.457449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.457726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.457994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.458226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.458577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.458908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.469802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.471331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.478034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:49.479430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:49.487953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:49.488908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:49.490126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:49.490126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:49.490126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:49.490126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 372/747] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:49.490410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.536342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.557633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 376/748] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:07:49.558577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.599722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.610638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d53d7-ede2-47de-a46b-1603747900ef None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.646114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.646731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.647009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.647274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.647924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:49.648221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.649614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.663009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 373/749] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:49.663633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.680889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc532e0d-501b-422e-a1f3-7b6a0da72ba6 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.718921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.719806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.720082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.720352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.730421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.731851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:49.749446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:49.750695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:49.751715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:49.751715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:49.751715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:49.751715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 377/750] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:49.755688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.776065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7d7e3-b9c1-4f31-8c61-b491ab49b01c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.790681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.815131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.815756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.816199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.816481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.817619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:49.818069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.819985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.832986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 374/751] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:49.834080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:49.853753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:49.864256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec3587-e1ad-4647-9055-60ab33a4603a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.909212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.910566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.910852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.911129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.911771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.913154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.939509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea346a-e951-44eb-a7ac-64a784ba9005 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:49.973814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:49.974435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:49.974711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:49.974976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:49.981894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:49.984097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:49.996021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 378/752] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:07:49 2022] POST /identity/v3/services => generated 240 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:49.996021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:50.003068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:50.004100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:50.005079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:50.009046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 375/753] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:50.010009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.088826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:50.120884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2164847-55ba-4be2-bbea-bb2c7809fbf4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.134573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:50.155854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b494b-f3c3-4eaf-9397-6c0645366a89 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.163013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.163908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.164328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.164777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.174499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.176195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:50.190242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:50.191364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 376/754] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:50.192408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.215590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.217212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.218136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.218512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.219336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:50.219725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.221219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.235956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 379/755] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:50.236959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.323971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d9299-fffe-4ccd-936c-789cbdea7b87 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.358752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039566f-4754-4c13-acc3-b904f370b303 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.365665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:50.370982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.371765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.372186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.372598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.377032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:50.377032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.377032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.394560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 377/756] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:50.395475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.404959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.405768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.406510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.406892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.408011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:50.408812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.410630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.429408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 380/757] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:50.431289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.446582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:50.516828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82687b-1029-488a-8260-7e7d4b393874 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.555550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.556183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.556466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.556741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.566303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.567925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.571578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7fd79-9bec-4164-9ce9-451746b0f5d8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:50.582103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:50.583137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:50.584193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 378/758] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:50.584761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.618460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.619423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.619804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.620406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.637984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.640656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:50.660806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:50.661935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:50.662955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:50.662955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:50.662955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:50.662955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 381/759] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:50.663222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.673531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:50.740518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc06b43-70b1-4e78-95f0-98ddb9c22c98 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.780963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.781893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.782323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.782742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.783628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=1bb11e8a7a124630bf9b0e611324fabf) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:50.784088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.785795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.792636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 379/760] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/role_assignments?user.id=1bb11e8a7a124630bf9b0e611324fabf&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:50.793725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:50.866969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:50.910514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897c9fc-3e49-4960-88c2-96c8241ff9ce None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:50.950570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:50.951421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:50.951794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:50.952779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:50.953820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:50.954380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:50.957042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:50.975412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 382/761] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:50.981666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:51.062946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f44384-48d1-4895-b2e1-c88c9711cd7f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:51.091348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:51.130763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:51.131384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.131651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.131908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.132557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:51.132861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:51.134259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:51.161897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 380/762] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:51.163742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8528adf-be04-4ab0-a156-05cead1fcbf0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:51.166255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:51.202274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:51.228763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:51.229406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.229948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.230226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.230962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=d0559bf2dae143468a9d25c82770a337) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:51.231272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:51.232710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:51.251354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 383/763] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:50 2022] GET /identity/v3/role_assignments?user.id=d0559bf2dae143468a9d25c82770a337&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:51.347172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0553e0-5362-4bdf-b423-39ddb285bdf3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:51.400260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:51.400925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.401281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.401557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.414060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:51.420303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c4e1bf90-de4e-4ed6-ae35-543a0a65f713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:51.427903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:51.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:51.430075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 381/764] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:51.435624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:51.521872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751f296-588b-4aba-a160-c7ddc443c2a5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.522151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751f296-588b-4aba-a160-c7ddc443c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.522813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751f296-588b-4aba-a160-c7ddc443c2a5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.524664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 382/765] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:51.534039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.534448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.534825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.544930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd436-bcc1-4627-9813-f75638990355 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:51.553683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:51.555019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:51.588535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:51.589173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.589413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:51.589503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.589704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.590449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:51.590689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:51.592100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:51.615126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 384/766] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:51.620819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:51.629535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 383/767] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:07:51.678279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:51.697775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:51.794315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f148099-2b41-4f69-8c2e-5e667c8f9a2c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:51.832571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:51.833725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.834214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.834718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/5dda95d65ff54916b0565d412c78e75f/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.836844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=5dda95d65ff54916b0565d412c78e75f, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:51.839084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:51.854353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3586cb9-8a03-4911-bb42-1bc2edcde41d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:51.892702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:51.893729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:51.894432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:51.894909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:51.895768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:51.897350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:51.907927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:51.908946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:51.909844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:51.912595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 384/768] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Aug 10 13:07:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:51.918095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:51.942210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 385/769] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:51 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/5dda95d65ff54916b0565d412c78e75f/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:07:51.960807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:52.073108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7230e-f31c-429c-a5b9-dc872f890332 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:52.123104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:52.123898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.124279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.124755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.125554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:52.125856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:52.127243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:52.137509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 385/770] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 10 13:07:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:52.144661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:52.266429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0e04-a636-4e5b-a229-ce2fd9bc1af5 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:52.268873 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:52.304324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:52.304937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.305270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.305542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.306233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:52.306596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:52.307953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f49b7-a428-45ce-a09d-d47fdbb3b5ab None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:52.317487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 386/771] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 10 13:07:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:52.326072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:52.438162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92bff8-6619-48f1-b307-e2d693d164d2 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.438357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92bff8-6619-48f1-b307-e2d693d164d2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.438634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92bff8-6619-48f1-b307-e2d693d164d2 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.445112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 387/772] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:52.454727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:52.454847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.454847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.454847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.472826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acca588-31e5-491b-b3b7-6abe1cbb1a3f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:52.487497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:52.491240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:52.537891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:52.538884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.539381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.539846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.540742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] RBAC: Adding query filter params (interface=public, service_id=50d495ae81d342e0b28de11719709669, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:52.541273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:52.543862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:52.561605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 386/773] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:07:52 2022] GET /identity/v3/endpoints?service_id=50d495ae81d342e0b28de11719709669&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:52.566941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:52.603352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 388/774] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:07:52.640888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:52.730710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de292599-4cca-46a1-a456-05da6e741fab None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:52.773795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:52.774428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.774685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.774957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.775561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:52.776910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:52.797590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 387/775] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:07:52 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:52.833447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:52.837641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d76251-334e-4797-b6e5-f74959d30802 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:52.893460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:52.894120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:52.894398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:52.894665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] PATH_INFO: `/v3/services/network` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:52.895256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:52.896640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:52.912241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:52.913319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:52.914161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 389/776] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:52 2022] GET /identity/v3/services/network => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:52.919651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:52.954268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:53.049995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627edf-d767-4a83-92bd-bcf50d6b2525 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:53.087019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:53.087978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.088488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.088978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.089855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:53.090374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:53.092079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c9cc1-ed6c-481f-90fd-8e8564f018d8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:53.098061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 388/777] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:07:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:53.104004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:53.176684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc97d0-8ea9-4a5f-9338-1b18841740a5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.177229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc97d0-8ea9-4a5f-9338-1b18841740a5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.177307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc97d0-8ea9-4a5f-9338-1b18841740a5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.178787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 389/778] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:53.184241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.184767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.185308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.204234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:53.205920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:53.239719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b34a5e-7356-4011-9da4-3754ea70d010 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:53.307589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:53.308187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.308430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.308643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.309303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] RBAC: Adding query filter params (type=network) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:53.309605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:53.310968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:53.323669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 390/779] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:07:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:53.331371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:53.355874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 390/780] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:07:53.382762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:53.386402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:53.431142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:53.524361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8b567-b608-43ac-83d9-704219a33a6f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:53.563252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cc85c-8c59-42e9-a70a-a7f505157305 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:53.585106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:53.586045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.586638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.586973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.587745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] RBAC: Adding query filter params (interface=public, service_id=2d9d6c6341a4478994ffe0235f71fb4f, region_id=RegionOne) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:53.588119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:53.589606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:53.601829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:53.602794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.603629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.604106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.604969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 391/781] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:07:53 2022] GET /identity/v3/endpoints?service_id=2d9d6c6341a4478994ffe0235f71fb4f&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:53.611725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:53.616464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:53.622626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:53.623541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:53.625006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:53.633155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 391/782] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:53.634625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:53.789363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:53.809120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:53.826950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ab657-ef5a-49e3-bc5c-8e6beac69e34 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:53.838499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481757a2-a0e5-48b0-86f6-f3d102118e59 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:53.867903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff05443-3af2-4e2e-8f43-bc20fd4e14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:53.868510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff05443-3af2-4e2e-8f43-bc20fd4e14d8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.868786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff05443-3af2-4e2e-8f43-bc20fd4e14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.869096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff05443-3af2-4e2e-8f43-bc20fd4e14d8 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.869848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff05443-3af2-4e2e-8f43-bc20fd4e14d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:53.871435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff05443-3af2-4e2e-8f43-bc20fd4e14d8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:53.878509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 392/783] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:07:53 2022] GET /identity/v3/services => generated 772 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:53.879731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cacf10-0147-4a14-8e2e-78db8b2dc1ff None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.880015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cacf10-0147-4a14-8e2e-78db8b2dc1ff None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.880289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cacf10-0147-4a14-8e2e-78db8b2dc1ff None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.885726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 393/784] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:53.887099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.887380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.887649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.890393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:53.891032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:53.891296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:53.891570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:53.892251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:53.892558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:53.893964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:53.918115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 392/785] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:53.919204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:53.921258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:53.923023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.056876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 394/786] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:07:54.079749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.083042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc557c-4fca-4e61-a359-487a2d8721cc None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.150340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.150994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.151272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.151549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.158984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:54.170238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.172237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:54.174617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:54.187424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e434c91-c36c-481c-bc6d-6eec95008b4e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:54.191240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:54.193654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:54.195936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:54.195936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:54.195936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:54.195936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 393/787] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:54.202571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.234504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.235741 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.236347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.236809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.252848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:54.259530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:54.260473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:07:54.261889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:54.262536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 395/788] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:54.270422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.341604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edceceb6-31d4-4b28-9ac2-7434c0f0da75 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.409399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.410121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.410407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.410678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.411332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:54.411635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.416088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:54.439082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 394/789] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:54.443688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.473579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5f585-d198-4b51-ac78-e0cd4767ac4e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.519877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:54.529163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.536281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.536281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.536281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.536281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:54.536281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.536281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a97f6-84ac-47e6-8014-642c257b5e99 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:54.550240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 396/790] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:54.550240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7859750-f64a-48f7-9b09-eda9e5332395 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.550240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7859750-f64a-48f7-9b09-eda9e5332395 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.550240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7859750-f64a-48f7-9b09-eda9e5332395 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.553067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 397/791] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:54.554152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7859750-f64a-48f7-9b09-eda9e5332395 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.567488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:54.595885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e4a72-43bd-4952-b0e1-87a9398b4d41 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.646311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.647959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.648951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.649453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.664793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.666266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1e2b0f3a-a590-477f-ba12-70923f33a35d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:54.678127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:54.679117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:07:54.680069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:54.684043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 395/792] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:54.685304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.685594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.685871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.695309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7859750-f64a-48f7-9b09-eda9e5332395 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.705512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:54.706945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:54.734871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.735876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.736486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.737117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.750220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.752123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:54.768678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:54.769715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:54.770627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:54.770627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:54.770627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:54.777438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 398/793] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:54.778198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.781417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 396/794] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:07:54.781417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.842587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:54.872619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:54.909954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26bf19-2717-4f60-aa79-3d5f325ea584 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.929862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2c25-bf04-4e39-80b6-e181afce6c6a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:54.952489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.953478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.953986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.954465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.955325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:54.955839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.957530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:54.974553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 397/795] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:54.974553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:54.977703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:54.978669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:54.979155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:54.979687 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:54.980630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:54.981188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:54.982842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.007834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 399/796] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:55.009167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.106247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.111914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf33bf-23d1-4b71-9fa2-12ea2e2fcea5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.118962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292ac1-20bc-405a-87a3-44e508c97a57 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.137535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.154200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.155120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.164276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.164276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.164505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.164505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.164505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.164505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.167159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:55.170425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.171882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:55.173314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:55.174276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 398/797] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:55.175140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-49a5c4fa-2f2c-4d6e-a2ad-e74e87d038cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:55.191024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:55.191999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:07:55.192929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:55.196804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 400/798] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:55.197997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31cb5c-ca6a-4403-a62d-e04f1c3f4bb0 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.198289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31cb5c-ca6a-4403-a62d-e04f1c3f4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.198567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31cb5c-ca6a-4403-a62d-e04f1c3f4bb0 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.201549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 401/799] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:55.201549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31cb5c-ca6a-4403-a62d-e04f1c3f4bb0 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.288179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d989cd7-5929-4df2-b8eb-5ec4bf0e8727 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.314742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31cb5c-ca6a-4403-a62d-e04f1c3f4bb0 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.331196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.332089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.332729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.333050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.348118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.349656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.350999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.351604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.351880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.352144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.352800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:55.353127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.354458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.363996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 402/800] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:55.364799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d6736aa2-9c05-44af-ba78-4ea03a6c854a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:55.367459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:55.368404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:55.369420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:55.369420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:55.369420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:55.373129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 399/801] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:55.375355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.375632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.375901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.394916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:55.396036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:55.399751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.418034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.470082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 400/802] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:07:55.471202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.482062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c5d83-03a4-4707-8148-c89fca55f91b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.521630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.522242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.522520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.522794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.523518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:55.523824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.525753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.549173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 403/803] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:55.549173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.589258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c9d06-ad36-427c-9762-9faf381a7002 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.625125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.625765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.626045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.626318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.634394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.636569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.638134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.646469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:55.653721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:55.654641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:55.655891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:55.655891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:55.655891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:55.655891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 401/804] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:55.655891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.663200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f25c-5a03-4e02-b983-d24bac1b2ae7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.704480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.705131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.705477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.705761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.706402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:55.706712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.708163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.729306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 404/805] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:55.730182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.770184 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8cb9d-5d0a-441d-8e86-40ed56033c72 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.807248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.808338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.808618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.808891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.809519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.812220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:55.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:55.822131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:55.822965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 402/806] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:55.823748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.839506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37b45-904e-4d98-85ba-e5eaeb42ba6d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.876066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.876694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.876975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.877271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.888674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.889342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.890910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:55.906466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:55.907425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:55.908316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:55.908316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:55.908316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:55.913362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 405/807] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:55.914113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.926490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.941945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b625fd-27d1-4211-b115-00d4837b2ce0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.979326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.979925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.980201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.980467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.981176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:55.981486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.982852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.000582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 403/808] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:56.000582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.025838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087cb8a-bcef-4117-a720-960ba7b0acc2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.063062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.063700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.063981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.064259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.071040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.072476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:56.084116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:56.085073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:56.085981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 406/809] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:56.086475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.113947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73f5e-e1e1-485b-8765-f3d22d418f3d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.133129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.152457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.165484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.165484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.165484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.165484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:56.165484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.165484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.165980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.166067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 404/810] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:56.166067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.198828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ab869-ce6e-4773-88e1-072062ec5ec8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.240807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.241713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.241984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.242252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.242888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:56.243341 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.244527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.262483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 407/811] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:56.262483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.271108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf8b2c-7719-4855-bf4d-3fbd1628d20d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.305782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.306425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.306702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.306977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.314661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.316107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:07:56.327556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:56.328503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:56.329492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 405/812] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:56.330099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.367409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2df64-26d1-4b70-b83b-233fe5f6d2c1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.370424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.393155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.401149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.401758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.402045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.402580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.403287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:56.403592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.405765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.419529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 408/813] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:56.419529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.428750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ab4e-3ff0-41a6-8d5b-02cf1a2388a5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.462267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.462875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.463158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.463572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.464200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:56.464501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.465884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b90d2-e558-4d5e-82bc-71844934fe5d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.471743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 406/814] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:56.472963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786447b-c90a-4612-ac9c-b6b79896b76f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.473276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786447b-c90a-4612-ac9c-b6b79896b76f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.473550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786447b-c90a-4612-ac9c-b6b79896b76f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.474890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 407/815] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:56.477178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786447b-c90a-4612-ac9c-b6b79896b76f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.521401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f94bf0-e4d1-49df-8b0a-784b9663f048 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.554991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.555625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.555906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.556181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.563335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.564849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:56.575837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:56.577069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 409/816] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:56.578281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.583180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786447b-c90a-4612-ac9c-b6b79896b76f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.585781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.617598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.618230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.618513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.619109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.619515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:56.619824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.621274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.628658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.638594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 408/817] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:56.639456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.685627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0148-691e-4691-8200-2950b2298096 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.723799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574d5ab-e50d-4dc1-94a4-15e1f6718caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.724459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d5ab-e50d-4dc1-94a4-15e1f6718caa None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.724737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d5ab-e50d-4dc1-94a4-15e1f6718caa None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.725045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d5ab-e50d-4dc1-94a4-15e1f6718caa None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/1bb11e8a7a124630bf9b0e611324fabf/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.726942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d5ab-e50d-4dc1-94a4-15e1f6718caa None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=1bb11e8a7a124630bf9b0e611324fabf, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.728771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d5ab-e50d-4dc1-94a4-15e1f6718caa None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.756573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9badc2-4343-4e0d-9d73-07dd5b18e97f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:56.790375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 410/818] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:56 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/1bb11e8a7a124630bf9b0e611324fabf/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:56.791685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.791982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.792261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.804333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:56.804975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:56.805288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:56.805561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:56.806240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:56.807756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:56.815580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:56.816820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:56.895688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 411/819] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:07:56.896374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.919013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 409/820] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:07:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:56.920560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:56.950180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:56.956673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:57.050362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591a022-9328-42a7-b4a7-24eedfe4081c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.097413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c14d31-2a96-4c7d-92fd-c44ad0535ce0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.099341 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.099972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.100260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.100538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.101279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:57.101589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.103171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.118149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 412/821] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:57.119160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.146366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.147437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.147964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.148438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.246165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.248375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.262308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:57.267890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d5e00-e27c-45f2-989a-c72265a4a233 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.271434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:57.314097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.314861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.315235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.315596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] PATH_INFO: `/v3/services/network` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.316527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.318053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:57.333603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:57.334682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:07:57.335527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:57.340942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 413/822] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:57 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:57.344006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.401206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 410/823] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:07:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:07:57.403043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.476161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae981f5a-4299-4bfb-85fe-deaf69404424 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.520234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.520914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.521228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.521503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/d0559bf2dae143468a9d25c82770a337/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.540268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c09c51-831d-4f2d-b920-4ba59399b1d8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.542935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=d0559bf2dae143468a9d25c82770a337, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.545291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.561593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:57.588006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.588971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.589450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.589887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.590680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] RBAC: Adding query filter params (name=network) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:57.591146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.593644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.598903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 411/824] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:07:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:57.599935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.621691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 414/825] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:07:57 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/d0559bf2dae143468a9d25c82770a337/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:07:57.622700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.665458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:57.822048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3e734-93a8-4f72-954c-174b8da30507 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.841517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb510c-45eb-4b55-953c-1c1ea65e0d79 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:57.881932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.883070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.883544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.884034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.893886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:57.894493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:57.894772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:57.895041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:57.895814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] RBAC: Adding query filter params (type=network) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:57.896082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.897474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.898267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:57.900266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:57.903342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 415/826] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:07:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:57.911937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:57.919715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:57.921277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:57.922167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:57.922167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:57.922167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:57.922167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 412/827] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:57.930033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:57.971855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:58.021401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:58.050754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f395e-2aa9-4fe5-b139-5414bb495a7e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:58.089144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:58.090106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:58.090659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:58.091194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:58.092163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:58.095249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:58.115752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae233bff-5c7b-407c-931a-5ffcec1eebe8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:58.148982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:58.149673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:58.149950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:58.150217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:58.150911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:58.151212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:58.152978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:58.166497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 413/828] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:58.173188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:58.191085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 416/829] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:07:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:07:58.285854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:58.296107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de9d77-4cc5-4647-9935-1eb739b291fe None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:58.339946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:58.340831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:58.341282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:58.341696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:58.349818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:58.351458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:07:58.364202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:58.365329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:07:58.366498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:58.368531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 414/830] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:58.375226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:58.491049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aa4be-6e04-4fb4-a447-d0536cf476be None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:58.523516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:58.530748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:58.531736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:58.532174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:58.532598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:58.533513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:58.533975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:58.535562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaf530-1840-4ba0-b625-0d97657502d8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:58.549303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 417/831] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:58.555539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:58.676471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4bd24-ad21-47a4-ab14-f0e39fb2e8b4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:58.715513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:58.716368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:58.716794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:58.717386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:58.718390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=5dda95d65ff54916b0565d412c78e75f) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:58.718872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:58.720436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2b727-7b8a-434b-bd47-88b45cb0ee02 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:58.727227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 415/832] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:07:58 2022] GET /identity/v3/role_assignments?user.id=5dda95d65ff54916b0565d412c78e75f&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:59.083610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbb1b7-e1ab-456a-a9a9-962fca9c021c None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.084011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbb1b7-e1ab-456a-a9a9-962fca9c021c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.084203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbb1b7-e1ab-456a-a9a9-962fca9c021c None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.085809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 418/833] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:07:59.091250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.091584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.091808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.109999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:59.111190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:59.114070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:59.141088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16695d8-9648-437d-9067-3d91bd94fcf4 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.141404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16695d8-9648-437d-9067-3d91bd94fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.141661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16695d8-9648-437d-9067-3d91bd94fcf4 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.143181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 419/834] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:07:59.147985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.148272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.148546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.171549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:07:59.173034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:59.196684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 416/835] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:07:59.220673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.248468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 420/836] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:07:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:07:59.271760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.331317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020facb-7da9-4a16-8943-0b9665aac0da None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.368362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.368919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.369230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.369504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.380843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:07:59.385785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:59.386730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 417/837] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:59.392647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:59.399544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77c138-2710-4a74-b12f-c902db9fcd19 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.399642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.419421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:59.434842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.435665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.436159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.436376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.436899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:59.438925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:59.450225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:59.451237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:07:59.452801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:07:59.457450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 421/838] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:07:59.458316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.525464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f0455-cefc-478a-9290-fe94545e6dc1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.568875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.570278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.570743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.571576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.572680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:59.573178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:59.575194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:59.586212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b04c7-5745-4b4e-8ecd-dbe96733d8d8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.591573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 418/839] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:07:59.594027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.635177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.636145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.636632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.637132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.638037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:59.638559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:59.640901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:59.647931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 422/840] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:59.656353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.679841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:59.722330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb37cd-dd23-466f-8418-37aaf3449b2e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.725374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:59.758855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.759501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.760088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.760088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.771711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:59.773204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:59.785371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba716da6-68b4-43d4-88b9-b451f65d0841 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:07:59.788740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:07:59.789707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:07:59.790793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:07:59.790793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:07:59.790793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:07:59.794560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 419/841] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:07:59.796078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:59.825489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.826467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.827137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.827744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.828600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:59.829146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:59.830715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:59.841152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 423/842] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:59.911516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f542f7-eaf1-4bb0-975b-f51e42abc418 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:59.927813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:59.950488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:59.962456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:59.962456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:59.962456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:59.962456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:07:59.962456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:59.962456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:59.973203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 420/843] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:07:59.974971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.087673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8bc36-743a-49e3-8d1f-296e20e37376 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.101880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:00.123281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.124177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.124460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.124749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.132601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:00.134237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5eb3edf4-7c52-4d3c-8981-ad7d664f3e6a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:00.146431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:00.147424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:00.148360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 424/844] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:07:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:00.153953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.182406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebee81f-d494-4484-9240-782aaedd8b97 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.182594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebee81f-d494-4484-9240-782aaedd8b97 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.182877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebee81f-d494-4484-9240-782aaedd8b97 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.184567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 425/845] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:00.189568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.189851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.190125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.208452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:00.209696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:00.263088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78ad20-5e2b-47e9-a30b-2654f45f29c1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.293096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 426/846] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:00.311819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.312622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.312899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.313201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.313926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:00.314229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:00.315769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:00.316007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.340518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 421/847] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:00.345851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.393139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:00.439489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed116-bab9-4fd4-95c5-c17f01157c20 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.442271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:00.481288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321741-66e4-48a9-820e-97ee37791424 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.488742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.489732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.490241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.490719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.505388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:00.512448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:00.513417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:08:00.514306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:00.517074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 427/848] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:00.521464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.528435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.528737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.529071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.529420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.612528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:00.614603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:00.654055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6c193-f45e-491d-baea-31be0b57a78e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.658827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:00.703953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.704483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.704766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.705064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.705739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:00.706477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:00.707859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:00.719339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 428/849] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:00.726828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.741792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 422/850] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:00.746866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.768117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:00.869767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33605-42d1-4112-8b56-fa340e8cf88e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.891633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce3aa-1ada-40bf-a4db-701b0e4b5992 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:00.919263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.921591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.922366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.922848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.924649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:00.930151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:00.930790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.931068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.931337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.938810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:00.950676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:00.950825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:00.950825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-f6995ee8-0897-4ff6-a954-40e31abfe8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:00.958127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:00.959065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:00.960023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:00.960023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:00.960023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:00.960199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:00.961674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:00.962548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:00.962548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:00.962548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:00.962712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 429/851] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:00.962780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 423/852] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:00.965140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd78dd-8b63-4157-b290-9fa7e11274a8 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:00.965140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd78dd-8b63-4157-b290-9fa7e11274a8 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:00.965140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd78dd-8b63-4157-b290-9fa7e11274a8 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:00.965410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:00.966001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 430/853] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:00.966719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd78dd-8b63-4157-b290-9fa7e11274a8 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.058820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.082453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd78dd-8b63-4157-b290-9fa7e11274a8 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.084724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247dd951-813a-4b57-9c3f-dcc4c22c1b0e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.121661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.122267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.122549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.122822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.123513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:01.123645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.123847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.124426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.124780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.125151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.125416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c87bd-c87d-4048-9997-8499f0e2b0d5 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.125921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:01.126352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.128189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.138033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 431/854] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:01.144248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.144534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.144810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.150839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 424/855] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:01.150839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.168093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:01.169311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:01.201351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.241769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 432/856] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:01.242533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.272867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f837-daa9-42f5-a3a9-dc9484fad88d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.309512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.310231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.310546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.310817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.317907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.318438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.319622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:01.331397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:01.335696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 425/857] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:01.336617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.359935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13b27e-709e-4e36-8941-bd88b7494912 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.395654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.396396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.396747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.397182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.405433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.407077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:01.418908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:01.419921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:01.421456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 433/858] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:01.422655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.447714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cdb1f-3a7d-4b3f-acd9-c8a46a3bfff3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.451100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.485516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.487060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.487546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.488000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.489056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.491028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:01.500256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:01.501230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:01.502159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:01.506393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 426/859] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:01.507082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.534952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967090d-c722-4402-bf14-c93bdeaab843 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.562487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.572012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.572619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.575625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.575625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.575625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:01.575625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.575625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.593956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 434/860] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:01.593956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.616296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a39a8-4f34-4b63-b84a-b4dfc33ec681 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.664578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.665259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.665544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.665826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.666559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:01.666870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.668284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.685813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 427/861] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:01.686741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.706527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2fc2-baf6-4d9d-a2ef-2befa57866e7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.717757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.743465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.744084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.744367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.744644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.745320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:01.745627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.746994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.751758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 435/862] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:01.757020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.791232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33d20a-87b2-4b43-9d05-f6738eddda82 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.813981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.826330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.827477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.827759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.828034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.828907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=1bb11e8a7a124630bf9b0e611324fabf) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:01.829249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:01.830670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:01.836921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 428/863] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/role_assignments?user.id=1bb11e8a7a124630bf9b0e611324fabf&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:01.837737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:01.872596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036871-4e84-4e81-9f7e-345235118732 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:01.919659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:01.920534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:01.920917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:01.921319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:01.984173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:01.999769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a051f9-cc21-4c41-aa05-f08872988970 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:02.000337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.002023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.045071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:02.045996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.046421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.046698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.047329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:02.047626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.048979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bfbe6-ec05-4677-86ce-f7ebbe54394c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.054368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 429/864] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:02.055641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2d528-19eb-4845-81da-cde68a2a33ee None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.056035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2d528-19eb-4845-81da-cde68a2a33ee None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.056311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2d528-19eb-4845-81da-cde68a2a33ee None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.057796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 430/865] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:02.064160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db00cfa-830f-402f-a7da-06720502949b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.064515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db00cfa-830f-402f-a7da-06720502949b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.064714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db00cfa-830f-402f-a7da-06720502949b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.074984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:02.086595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2db00cfa-830f-402f-a7da-06720502949b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:02.088084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2db00cfa-830f-402f-a7da-06720502949b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:02.150261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 436/866] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:02.157418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:02.188573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 431/867] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:02.212739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db00cfa-830f-402f-a7da-06720502949b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:02.284325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b72e-49f6-48b2-add9-f655a6dc8aa4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:02.323251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:02.323892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.324174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.324446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.325340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:02.339800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.341288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.343555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db00cfa-830f-402f-a7da-06720502949b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:02.356735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:02.358416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:02.359664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:02.359664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:02.359664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:02.359664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 437/868] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:02.363909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:02.373036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:02.391873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:02.392771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.393204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.393601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.394528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.396657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.473038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 432/869] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:02 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:02.485319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904701-793f-4402-b6da-28ddc0efb9ce None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:02.532058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:02.532691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.532971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.533314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.533997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:02.534386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.535974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.551978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 438/870] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:02.558116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:02.674687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600bc00-5325-4b57-9f3d-1c3247df7f88 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:02.711223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:02.711911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.712198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.712514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.720328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.721794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:02.734113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:02.735038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:02.736401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 433/871] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:02.743039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:02.806546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:02.855545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901b537-5fa4-4872-81cd-7c7e76e3858b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:02.907674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:02.908298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:02.908585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:02.908861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:02.909595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:02.909903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:02.911348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5eec62-2d6e-4a9c-b882-118cd5481f46 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:02.923909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 439/872] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:02.930119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:02.934270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:03.036930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbc0f4-64ba-4de2-b383-cbdccdaf17bb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:03.073623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:03.074565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:03.075045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:03.075529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:03.076447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=d0559bf2dae143468a9d25c82770a337) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:03.076982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:03.078630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e7d0a-fc02-460b-b279-82a806306ef5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:03.089127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 434/873] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:02 2022] GET /identity/v3/role_assignments?user.id=d0559bf2dae143468a9d25c82770a337&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:04.039239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e0e3f-0280-4159-a293-505f0a6fba03 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.040281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e0e3f-0280-4159-a293-505f0a6fba03 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.040773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e0e3f-0280-4159-a293-505f0a6fba03 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.042632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 440/874] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:04.048773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73adef-b261-4f79-ab5f-dc8d8cc6ebb1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.049612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73adef-b261-4f79-ab5f-dc8d8cc6ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.050101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73adef-b261-4f79-ab5f-dc8d8cc6ebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.069026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9c73adef-b261-4f79-ab5f-dc8d8cc6ebb1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:04.070504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9c73adef-b261-4f79-ab5f-dc8d8cc6ebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:04.136097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:04.143605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 435/875] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:04.165712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e0e3f-0280-4159-a293-505f0a6fba03 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:04.192832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9612f-5c96-4765-a99f-00ed22324c5a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.193147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9612f-5c96-4765-a99f-00ed22324c5a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.193417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9612f-5c96-4765-a99f-00ed22324c5a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.194986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 436/876] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:04.199777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.200060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.200332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.221181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:04.222818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:04.276668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e0e3f-0280-4159-a293-505f0a6fba03 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:04.297448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 437/877] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:04.313388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:04.321884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:04.325988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:04.326959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.327391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.327864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.328652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:04.330354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:04.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:04.340714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:04.341594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:04.344452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 441/878] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:04.350206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:04.449411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6b95b-0983-4249-bd3f-a2d1bac10cf4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:04.482325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c6329-39c4-4f1a-bc07-dd4f99dbefcc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:04.483261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:04.493236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcecde1-7892-4ef4-b9cd-a3ffa4aab202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:04.494381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcecde1-7892-4ef4-b9cd-a3ffa4aab202 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.494776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcecde1-7892-4ef4-b9cd-a3ffa4aab202 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.495146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcecde1-7892-4ef4-b9cd-a3ffa4aab202 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.496197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcecde1-7892-4ef4-b9cd-a3ffa4aab202 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:04.497927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcecde1-7892-4ef4-b9cd-a3ffa4aab202 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:04.523935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:04.524546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.524840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.525131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.525815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:04.526127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:04.527817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:04.537300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 442/879] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:04.538271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:04.591490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:04.641022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 438/880] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:04 2022] POST /identity/v3/services => generated 269 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:08:04.641022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9788f-8f47-46aa-b193-ac8cc56e363d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.641022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9788f-8f47-46aa-b193-ac8cc56e363d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.641022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9788f-8f47-46aa-b193-ac8cc56e363d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.641022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 439/881] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:04.641880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.642166 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.642445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.660007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbcea5-0fe2-49d8-be64-5a07c2b68abf None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:04.667451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:04.668737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:04.705527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:04.706150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.706432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.706757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.707407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:04.707711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:04.709126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:04.714795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 443/882] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:04.755972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 440/883] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:04.780445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:04.906787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd997bd-d8e9-4b99-b03e-c6503549a593 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:04.917394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:04.946059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:04.946703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:04.947039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:04.947318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:04.947912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:04.949293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:04.958502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:04.959551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:04.960407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 444/884] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:08:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:04.965733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:05.073672 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8698d0a-ccd0-41fc-8b87-be316b4f1de2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:05.093038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:05.110562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:05.111181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.112702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.112702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.112702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:05.112702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:05.113994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cbfe5-a438-4c90-b6a8-65adc1b8330b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:05.119311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 441/885] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:08:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:05.125475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:05.234119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58153548-cc57-4526-9895-c71e15fe4583 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:05.266370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c7e3-7e27-4b2f-ad7d-16c73ac79f5e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.266696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c7e3-7e27-4b2f-ad7d-16c73ac79f5e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.266919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c7e3-7e27-4b2f-ad7d-16c73ac79f5e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.268392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 442/886] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:05.273032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:05.273652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.273933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.274203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.274534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.274839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:05.275136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:05.275425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.275895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.276501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:05.285037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 445/887] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:08:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:05.289996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:05.298921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:05.300404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:05.335824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:05.404378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 443/888] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:05.437019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:05.437216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7a76-700d-42b6-9703-93eafcc4734b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:05.485797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:05.486346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.486635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.486916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.487635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] RBAC: Adding query filter params (interface=public, service_id=47f4e22678314bdabdaa9e862b5bceb5, region_id=RegionOne) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:05.487947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:05.489372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:05.499322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 446/889] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:08:05 2022] GET /identity/v3/endpoints?service_id=47f4e22678314bdabdaa9e862b5bceb5&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:05.505499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:05.572282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75866f95-0384-4c1c-bbd7-8b91b6172711 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:05.597407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:05.609705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:05.610660 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.616141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.616141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.622070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:05.627408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:05.628423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:05.629645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:05.631937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 444/890] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 10 13:08:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:05.638711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:05.765378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:05.769103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b1214-47e7-436b-9dd7-6e33065e7589 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:05.785624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f2eb6-7ef7-4377-8a5a-30d79314a09e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:05.838823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:05.840831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.842990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.843419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.844395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:05.846918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:05.859023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:05.860407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 447/891] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:08:05 2022] GET /identity/v3/services => generated 1268 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:08:05.865878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:05.866180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:05.866439 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:05.867138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:05.867446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:05.868850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:05.888661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 445/892] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:08:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:05.895515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:06.014692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e161db-211c-4563-8478-51b29aebdd0d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:06.052251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:06.053397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.053397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.053397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.068481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:06.070255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:06.085727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:06.086706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:06.087609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:06.087609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:06.087609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:06.089780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 448/893] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:06.095869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:06.133222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:06.209377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a761b-3a03-4e41-8c41-c84a9bcf62bb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:06.246097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:06.246717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.246999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.247275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.247931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:06.248235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:06.249761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:06.262657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 446/894] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:06.268665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:06.292189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:06.378524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91c7f6-69a8-4175-8000-ead5c302a9de None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:06.414921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:06.416000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.416000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.416160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.424881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:06.426370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:06.438639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:06.439610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:06.440556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:06.444482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 449/895] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:08:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:06.445979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:06.546621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:06.553900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e265dc3-0d4f-4e5d-aebf-c53b79ac9142 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:06.590411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:06.591032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.591316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.591589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.593103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:06.593362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:06.594809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddec6-7dc8-40dc-83a1-709d964a73d4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:06.613372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 447/896] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 10 13:08:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:06.618953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:06.713926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:06.737843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367476-b0f5-48db-9ffb-8f31e4574ea1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:06.773638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:06.774580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.775008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.775475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.847919 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:06.849899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:06.850451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91f25a-f45f-43fb-8b25-b971cef82b9f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.850739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91f25a-f45f-43fb-8b25-b971cef82b9f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.851017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91f25a-f45f-43fb-8b25-b971cef82b9f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.852622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 448/897] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:06.857740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448823-95f0-4321-8bd7-fa75962f607b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.858237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448823-95f0-4321-8bd7-fa75962f607b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.858468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448823-95f0-4321-8bd7-fa75962f607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.876324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-75448823-95f0-4321-8bd7-fa75962f607b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:06.877607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-75448823-95f0-4321-8bd7-fa75962f607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:06.929645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:06.949794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 449/898] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:06.951058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5fc75-f434-4b6d-99d0-af8fb0ffe81a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.951937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5fc75-f434-4b6d-99d0-af8fb0ffe81a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.952403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5fc75-f434-4b6d-99d0-af8fb0ffe81a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.953999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 450/899] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:06.957954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 450/900] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:06 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:06.959363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:06.959646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:06.959835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:06.959994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:06.979259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:06.980559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:07.061024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 451/901] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:07.061024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.082649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.086354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df87fc-1744-41d6-a6e1-aafff6ee29c0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.122421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.123060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.123417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.123695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.134115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.135560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:07.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:07.151991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:07.153193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:07.153193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:07.153193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:07.153193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 451/902] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:07.153193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.180589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5a5bc-eaf3-4910-811e-af8535b3e39c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.185352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.218825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.219457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.219734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.220007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.220599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.222155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:07.230738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:07.231670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:07.232549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 452/903] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:07.233452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.265806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdc0d3-e02c-4c87-8a57-ebf8270c816f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.301353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.301978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.302225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.302495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.303136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.304481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.319092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.340868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0e6a-cfcb-453e-8d04-24c3799de44a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.368073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 452/904] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:07 2022] POST /identity/v3/services => generated 241 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:07.380392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.383419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.384414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.384699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.384972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.385669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:07.387097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.388617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.409200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 453/905] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:07.413517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.503736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.517900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435838-6190-4a2a-84ae-8c02a55b7185 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.550210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947dad9-f59a-420c-80d3-fef5dada040d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.559449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.563729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.563729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.563729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.563729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:07.563729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.563729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.569649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 453/906] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:07.575553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.589473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.595346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.595987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.596486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.596659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.609301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.611081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:07.627204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:07.628117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:07.629021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:07.631391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 454/907] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:07.637256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.699493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b06eb-4297-4c40-9a15-c6530bfe35e2 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.737897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.738990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.739514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.739980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.740805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:07.741334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.742885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.750117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 454/908] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:07.761083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.772358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab143eb3-d38a-4b24-8803-cec6de018ec8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.787241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.818243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.819451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.820828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.821271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.822550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:07.823495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.825335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.847192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 455/909] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:07.848806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:07.898549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ead0f2-8e27-4879-b3a6-644d5d395995 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.907269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:07.949197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:07.950167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:07.950684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:07.951150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:07.952279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] RBAC: Adding query filter params (interface=public, service_id=b2527a0358bf4c8492f834b879b2ac72, region_id=RegionOne) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:07.952785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:07.954378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:07.960189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64929-4a64-4b37-a2dd-8a9b94dac0c2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:07.960639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 455/910] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/endpoints?service_id=b2527a0358bf4c8492f834b879b2ac72&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:07.966503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:08.005210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:08.006733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.007016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.007286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.008019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] RBAC: Adding query filter params (role.id=c9b6e50662504ff9922f4831d6cf4490, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=d0559bf2dae143468a9d25c82770a337) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:08.008328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:08.009779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650cb87-0a0b-4451-a38d-df60214edcb2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:08.017066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 456/911] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/role_assignments?user.id=d0559bf2dae143468a9d25c82770a337&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=c9b6e50662504ff9922f4831d6cf4490 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:08.018437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a98f60-9b56-4738-a031-e9bcf9ef45f8 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.018727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a98f60-9b56-4738-a031-e9bcf9ef45f8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.018998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a98f60-9b56-4738-a031-e9bcf9ef45f8 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.020326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 457/912] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:08.025943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.026139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.026372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.046465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:08.047702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:08.072576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:08.098991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c53ef9-fba6-40bd-b9c0-021ab870562f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:08.133556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 458/913] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:08.138759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:08.139842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.140332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.140811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.141697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:08.143314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:08.150882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 456/914] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:08:07 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:08:08.151683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:08.208549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:08.272023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d92e9-bb00-4b85-931d-0fd826a0175d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:08.312028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:08.313058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.313546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.314024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.314822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:08.316185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:08.325317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:08.326278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:08.328327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 459/915] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:08:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:08.334099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:08.444690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aa7d2-4e89-4913-9ad2-5c95b696dc7b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:08.481278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449159f-406a-4387-8446-c030425324d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:08.482247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449159f-406a-4387-8446-c030425324d7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.483300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449159f-406a-4387-8446-c030425324d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.483952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449159f-406a-4387-8446-c030425324d7 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.484779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449159f-406a-4387-8446-c030425324d7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:08.485308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449159f-406a-4387-8446-c030425324d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:08.486855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449159f-406a-4387-8446-c030425324d7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:08.492404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 457/916] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:08:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:08.502026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:08.612251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31c444-0256-4860-b959-de2fc625c752 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:08.624201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:08.656034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:08.657269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.657356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.657617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.658256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:08.658566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:08.660204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b2f58-3590-4a21-9742-9004131c7876 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:08.666304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 460/917] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 10 13:08:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:08.672177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449159f-406a-4387-8446-c030425324d7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:08.689162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:08.782928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449159f-406a-4387-8446-c030425324d7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:08.817723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea6ec7-ab22-4228-aabc-911e2c2aa7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:08.818644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea6ec7-ab22-4228-aabc-911e2c2aa7db None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:08.819148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea6ec7-ab22-4228-aabc-911e2c2aa7db None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:08.819651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea6ec7-ab22-4228-aabc-911e2c2aa7db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:08.820472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea6ec7-ab22-4228-aabc-911e2c2aa7db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:08.822044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea6ec7-ab22-4228-aabc-911e2c2aa7db None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:08.902673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 458/918] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:08:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:08:09.569754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc240ffc-c369-4691-b8a0-2a6e7bfcaa72 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:09.570763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc240ffc-c369-4691-b8a0-2a6e7bfcaa72 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:09.571246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc240ffc-c369-4691-b8a0-2a6e7bfcaa72 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:09.573128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 461/919] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:09.578813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:09.579241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:09.579420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:09.598439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:09.599610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:09.673427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:09.683851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 459/920] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:09.708920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc240ffc-c369-4691-b8a0-2a6e7bfcaa72 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:09.823245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc240ffc-c369-4691-b8a0-2a6e7bfcaa72 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:09.858383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:09.859076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:09.859350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:09.859620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:09.860200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:09.861583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:09.870647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:09.871775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:09.872688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 462/921] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:09.878960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:09.981267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517658e-ba8e-440d-9424-66f496acffa9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.017164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.017786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.018061 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.018332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.018983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:10.019312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.020684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098513-83c1-4f2f-9194-950a65b03eff None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.026091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 460/922] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:10.034095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.082793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.146063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d9c0e-a226-478d-b0f8-3dd6b6cc32e1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.180944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.181577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.181854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.182122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.182744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:10.183101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.184544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.188917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac47087-a8db-4f2c-bf74-a2a443f3899c None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.189207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac47087-a8db-4f2c-bf74-a2a443f3899c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.189475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac47087-a8db-4f2c-bf74-a2a443f3899c None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.190973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 461/923] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:10.194108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 463/924] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:10.196302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.196660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.201035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.202891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.215995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:10.217882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:10.269346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.288790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 462/925] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:10.314633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.334494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6fe7-f2ca-4229-b61c-aa2cdf8c4196 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.335914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.370020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.370638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.370926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.377140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.377140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] RBAC: Adding query filter params (interface=public, service_id=c27b032b63354c89bd8354c73472a3be, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:10.377140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.377140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454295f1-7791-425d-aae7-dc5b69baaf6c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.380141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 464/926] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/endpoints?service_id=c27b032b63354c89bd8354c73472a3be&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:10.383064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3330a-731b-4b1f-874e-d91f59b31ca3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.383064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3330a-731b-4b1f-874e-d91f59b31ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.383064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3330a-731b-4b1f-874e-d91f59b31ca3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.383346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 465/927] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:10.386057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3330a-731b-4b1f-874e-d91f59b31ca3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.431158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326c406-8848-4772-a01e-c9291e216aa3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.468861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0e6fe-2de6-46fa-9673-fae7aee88aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.469537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0e6fe-2de6-46fa-9673-fae7aee88aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.469821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0e6fe-2de6-46fa-9673-fae7aee88aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.470095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0e6fe-2de6-46fa-9673-fae7aee88aa4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.481931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0e6fe-2de6-46fa-9673-fae7aee88aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-c7b0e6fe-2de6-46fa-9673-fae7aee88aa4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:10.487352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:10.488327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 463/928] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:10.489995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.490779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.490779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.502140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3330a-731b-4b1f-874e-d91f59b31ca3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.508870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:10.510106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:10.543482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.544090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.544375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.544635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.545951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.547522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.553512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 466/929] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:08:10.554080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.558416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.593103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 464/930] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:10.616007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.622842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.712668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d80883-118d-4979-9faa-fa1aece1d55b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.755268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc05f7-361c-4dbd-8dc9-8ddf93ba2d6e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.763400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.764172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.764554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.764890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.765690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:10.766066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.767519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.777032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 467/931] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:10.785470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.801826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.802790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.803282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.803700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.804499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.806134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:10.816665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:10.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:08:10.818947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:10.821466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 465/932] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:10.832861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:10.883488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.890805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:10.892769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4d363-a0df-4b81-a3a3-5fbb0fe06f0f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.929563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:10.930400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.930754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.931103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.958574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:10.960111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:10.974641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9582-7cd0-4019-a7b2-6b6fbccd1f46 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2545dbc7-8eb3-4bfd-98dc-098e581af0c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:10.975340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:10.976310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:10.977245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:10.977245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:10.977245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:10.981076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 468/933] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:10.982331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a3a9-eee8-4a02-9865-534f1c0b866e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:10.982750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a3a9-eee8-4a02-9865-534f1c0b866e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:10.983373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a3a9-eee8-4a02-9865-534f1c0b866e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:10.984549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 469/934] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:10.985115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a3a9-eee8-4a02-9865-534f1c0b866e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.015164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.015774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.016056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.016329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.016958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:11.017292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.018668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7b8d-e320-4ce4-bff3-82e13f827e03 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.028002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 466/935] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:11.028002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.028002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.028002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.043754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:11.044932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:11.108547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a3a9-eee8-4a02-9865-534f1c0b866e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.121091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 467/936] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:11.122244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.150122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.150657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.150939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.151207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.151851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:11.152155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.153816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.157671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:11.172509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 470/937] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:11.172509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.174110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:11.235652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f8032-6ba8-497a-8e71-60adab3b8afb None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.277291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.278167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.278445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.278714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.278882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5c6f-65cc-4ed1-a642-49b81a9e935f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.279352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:11.280386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.281785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.289036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 468/938] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:11.289036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.321620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.322216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.322497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.322772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.323407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.324838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.328080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING keystone.server.flask.common [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 13:08:11.363464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.sql.core [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 13:08:11.363464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 13:08:11.363464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [parameters: {'id': '42531a6006a6403ab7db38eb0f0effc4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 13:08:11.363464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80228) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 42531a6006a6403ab7db38eb0f0effc4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 42531a6006a6403ab7db38eb0f0effc4. Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:08:11.370734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result.read() Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 13:08:11.371569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application yield resource Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise self.value Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application session.commit() Aug 10 13:08:11.378453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.session.flush() Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application compat.raise_( Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 13:08:11.379397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 13:08:11.380288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result.read() Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:08:11.381102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [parameters: {'id': '42531a6006a6403ab7db38eb0f0effc4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:11.382003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:11.382785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 42531a6006a6403ab7db38eb0f0effc4. Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 471/939] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Aug 10 13:08:11 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:11.383673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.407544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:11.413232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7a8b2-3531-4f03-9bed-277348557709 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.446300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:11.453570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.454217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.454496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.454773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.462061 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.463545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:11.475673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:11.476708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 469/940] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:08:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:11.478035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.494296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96772dfb-73c8-46ea-bc7c-7b8e2be2114f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.529696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.530320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.530720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.531085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.531860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.533451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.582245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32aa79-731d-487c-a53c-42fd431e03d4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.617201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.617875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.618179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.618506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.625210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 472/941] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:08:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:11.626056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.630383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.631912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:11.652118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:11.653451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:11.654373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 470/942] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:11.662495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.700426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:11.807161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9a613-e2f8-4181-9f11-a6d0ab3cf1f7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.810289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2dcad-3b9d-4cc3-8e90-c1d902266743 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:11.822103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:11.853678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.853905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:11.854000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.854092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:11.854342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.854455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:11.854535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.854605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:11.855161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:11.855322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:11.855470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.855667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:11.856885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.857104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:11.878196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 471/943] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:11.887127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 473/944] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 10 13:08:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:11.892694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:11.981264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:12.006498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28692664-c820-4688-82b3-ec31ed10211b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:12.047204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:12.047915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.048204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.048479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.059225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:12.060772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:12.076003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:12.077584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:12.079103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:12.079103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:12.079103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:12.079103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 472/945] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:12.083458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:12.192635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9da170-0616-43e3-9fc1-0e2edc146a65 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:12.227414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:12.228398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.228906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.229492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.230367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:12.230878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:12.233028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:12.245638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 474/946] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:12.253949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:12.356497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17c237-72e9-4578-a254-afad79252977 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:12.383960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:12.394062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:12.394761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.395043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.395320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.402811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:12.416554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:12.416901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:12.417834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:12.418728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 473/947] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:12.425983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:12.459426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:12.535272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0f9cd-64ea-4462-9335-761e8e37891e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:12.570901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:12.571566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.571852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.572124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.572811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:12.573146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:12.574559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a1c45-c21c-4644-b6a5-eb56668616c2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:12.587885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 475/948] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:12.593839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:12.670488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67bdc9-ff54-4b56-a978-298cfdaba891 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.670968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67bdc9-ff54-4b56-a978-298cfdaba891 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.671047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67bdc9-ff54-4b56-a978-298cfdaba891 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.672404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 476/949] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:12.677690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.677975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.678247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.699062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:12.701339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:12.703410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1dc64-ec06-439d-8c0e-58b4cb068dfc None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:12.740390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:12.741189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.741472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.741748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/d0559bf2dae143468a9d25c82770a337/roles/c9b6e50662504ff9922f4831d6cf4490` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.744138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=d0559bf2dae143468a9d25c82770a337, role_id=c9b6e50662504ff9922f4831d6cf4490)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:12.746228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:12.778936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 477/950] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:12.802386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:12.824233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 474/951] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:08:12 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/d0559bf2dae143468a9d25c82770a337/roles/c9b6e50662504ff9922f4831d6cf4490 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:08:12.833021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:12.922708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ae5e6-a027-4732-80c3-4103a4bd4b1d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:12.962035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:12.962680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:12.962958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:12.963228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:12.963815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:12.965217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:12.974927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:12.975992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:12.977008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 478/952] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:12.983039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:13.036078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:13.089817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f975c-9dc3-4010-b0b6-2e80e9d82db3 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:13.134613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:13.135221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:13.135499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:13.135767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:13.136398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:13.136697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:13.138095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:13.143222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 475/953] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:13.148798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:13.240002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:13.255616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd697b-5353-413e-9bf7-db2abd55a3a1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:13.289876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:13.290482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:13.290755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:13.291024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:13.291648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:13.291949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:13.293331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e3866-8924-4b35-a825-b47325590392 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:13.298589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 479/954] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:13.304238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:13.414750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9c5db-2bdb-4134-8e4c-b3a27b545fa7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:13.429370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:13.450957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b888d7-66a3-45cc-a3ab-66246faa3ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:13.452282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b888d7-66a3-45cc-a3ab-66246faa3ede None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:13.452564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b888d7-66a3-45cc-a3ab-66246faa3ede None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:13.452838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b888d7-66a3-45cc-a3ab-66246faa3ede None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:13.453535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b888d7-66a3-45cc-a3ab-66246faa3ede None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:13.454899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b888d7-66a3-45cc-a3ab-66246faa3ede None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:13.534966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 476/955] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:08:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:13.888567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370a61b-02e9-4716-b07d-3accfcc0d892 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:13.889894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370a61b-02e9-4716-b07d-3accfcc0d892 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:13.889894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370a61b-02e9-4716-b07d-3accfcc0d892 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:13.891297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 480/956] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:13.896437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2ce70-f33a-4796-92dc-eef9c5c489ad None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:13.896864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2ce70-f33a-4796-92dc-eef9c5c489ad None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:13.897085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2ce70-f33a-4796-92dc-eef9c5c489ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:13.915906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-47e2ce70-f33a-4796-92dc-eef9c5c489ad None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:13.917111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-47e2ce70-f33a-4796-92dc-eef9c5c489ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:13.994382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 477/957] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:14.016781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370a61b-02e9-4716-b07d-3accfcc0d892 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:14.098870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da895488-5991-4a93-b791-046ab9914471 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.099221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da895488-5991-4a93-b791-046ab9914471 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.099432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da895488-5991-4a93-b791-046ab9914471 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.102316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 478/958] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:14.106922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.107213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.107517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.127466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:14.128777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:14.131790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370a61b-02e9-4716-b07d-3accfcc0d892 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:14.169223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:14.169866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.170702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.170977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.171561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:14.173059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:14.182549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:14.183617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:14.184900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 481/959] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:14.189606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:14.209662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:14.217356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 479/960] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:14.237365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:14.268332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:14.319533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0171f0-b7b2-42c2-aee6-84903e01590d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:14.364580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b3e15-f4d0-46a8-ae8c-adc291ef3f4c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:14.364976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:14.365873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.366495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.367687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.368520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:14.369044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:14.370583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:14.376882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 482/961] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:14.382894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:14.409953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:14.411014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.411689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.412210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.413076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:14.414991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:14.417896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING keystone.server.flask.common [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 13:08:14.458959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.sql.core [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 13:08:14.458959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 13:08:14.458959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [parameters: {'user_id': 'f6325b1377c94ac7a2d52c550048bc02', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 13:08:14.458959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80227) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 13:08:14.479752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application result.read() Aug 10 13:08:14.481851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:08:14.483290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application yield resource Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise self.value Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application session.commit() Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 13:08:14.484274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self.session.flush() Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application compat.raise_( Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 13:08:14.485223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:08:14.494008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application result.read() Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:08:14.496054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f6325b1377c94ac7a2d52c550048bc02', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:14.498926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 13:08:14.501902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 480/962] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 10 13:08:14 2022] POST /identity/v3/users => generated 160 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:14.504325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:14.525097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f4600-491e-41e8-900a-70b6fad44c0a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:14.574030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:14.575236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.576014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.577383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.578520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:14.579053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:14.580922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:14.588075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 483/963] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:14.605894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:14.624460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87871a46-b567-4542-af9c-6ef576bb86ee None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:14.682665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:14.683409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.684489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.684489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.694376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:14.697091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a166a132-bb38-4e8d-a4c5-a943d6a75ffe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:14.718109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:14.719111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:08:14.720053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:14.720605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 481/964] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:14.730037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:14.845773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37cb6f-de75-4bd5-9e5b-8439d018180e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:14.883249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:14.883861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:14.884144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:14.884414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:14.885155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:14.885463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:14.887024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914b7b8-773e-43c9-a02e-c39ccaf4bbeb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:14.912539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:14.925701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 484/965] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 10 13:08:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:15.091330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a92e-ab7e-41d3-9c9a-4c20e56bdf6b None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.091838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a92e-ab7e-41d3-9c9a-4c20e56bdf6b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.092520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a92e-ab7e-41d3-9c9a-4c20e56bdf6b None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.096809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 482/966] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:15.102201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.102495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.102779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.126037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:15.127838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:15.212144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 485/967] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:15.233095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3a92e-ab7e-41d3-9c9a-4c20e56bdf6b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.339135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3a92e-ab7e-41d3-9c9a-4c20e56bdf6b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:15.375219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:15.376120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.376564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.377029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.387985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:15.393833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:15.394929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:15.395942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:15.397950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 483/968] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:15.405026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.505793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ab16-2b4a-4a51-bd27-7e051113a2b3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:15.533755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:15.542062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:15.542682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.542959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.543232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.543900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:15.544204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:15.546239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ae38-6cda-4743-a384-f620b87de593 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:15.558825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 486/969] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:15.560354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.569808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:15.579164 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183c663-0be0-4171-b493-d1354fa4ffe2 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.579456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183c663-0be0-4171-b493-d1354fa4ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.579733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183c663-0be0-4171-b493-d1354fa4ffe2 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.583680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 487/970] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:15.586124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.586402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.586669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.605676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:15.607048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:15.672293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba876-ff87-4a25-84ed-a858a228a727 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:15.685099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 488/971] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:15.707354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:15.707621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.707988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.708271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.708617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.719663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:15.721145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:15.735963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:15.736928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:15.737871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:15.737871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:15.737871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:15.737871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 484/972] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:15.742647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.803522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:15.824408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b686c-dcf0-4ac0-bedd-6cc4426947e7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:15.846662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee796b0c-4be7-47c8-be74-6bfaaa39d929 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:15.849408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:15.864442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:15.865160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.865409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.865687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.866270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:15.867666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:15.876372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:15.877471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:15.878309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:15.882618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 489/973] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:15.883764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.885148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:15.885766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:15.886049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:15.886318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:15.886963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:15.887872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:15.889222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:15.901760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 485/974] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:15.907526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:15.995632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac25c6-b0b9-4499-8f3c-d128545a40a2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.012084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3dc08-a44c-49ac-bf5e-13caf7e37051 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.031553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.032072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.032682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.032962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.033269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.033892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:16.034194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.035540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.040559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 490/975] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:16.046823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.051094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.051724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.052003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.052270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.059340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.060969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:16.074069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:16.075109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:16.076763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 486/976] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:08:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:16.081346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.101142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.157123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3ec32-54ee-410d-a336-51f2b1ace8f9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.184080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0e48-2252-441d-b5d5-9fccd540ed38 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.196598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.197246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.197528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.197800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.198425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:16.198937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.200205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.205228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 491/977] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:16.211058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.226399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.227237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.227524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.227800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.228537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:16.228844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.230314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.254251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 487/978] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:16.255322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.294009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.320446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bfb88-4f0c-4ae4-9edd-4edafbfd4fa7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.334518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.356355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.356969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.357286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.358127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.359229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] RBAC: Adding query filter params (interface=admin, service_id=c27b032b63354c89bd8354c73472a3be, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:16.359538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.360899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.367481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 492/979] 10.222.0.40 () {56 vars in 1249 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/endpoints?service_id=c27b032b63354c89bd8354c73472a3be&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:16.370281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916351e0-0307-4b31-8f6f-892872721b14 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.375592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.409165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8573-7991-4dbf-840d-f0647f0ee853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.409777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8573-7991-4dbf-840d-f0647f0ee853 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.410059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8573-7991-4dbf-840d-f0647f0ee853 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.410334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8573-7991-4dbf-840d-f0647f0ee853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.482213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b8573-7991-4dbf-840d-f0647f0ee853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.484165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b8573-7991-4dbf-840d-f0647f0ee853 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.498793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d8e82-85a3-4ab0-b9f9-eb2fec284cc3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.525269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.541895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.542510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.542794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.543066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.543671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.545645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.551977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 493/980] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:08:16.598364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 488/981] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:16.604156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.655071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.709877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135c9a-7792-4542-bc67-b0096a83ca68 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.710168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135c9a-7792-4542-bc67-b0096a83ca68 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.710431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135c9a-7792-4542-bc67-b0096a83ca68 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.711792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 489/982] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:16.717718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.717718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.717718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.723094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b348180-ee51-4b2c-bdb8-3a024c7f7063 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.736980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:16.738788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:16.761383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.762022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.762297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.762566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.774591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.776027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:16.790914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:16.791916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:16.792802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:16.792802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:16.792802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:16.792802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 494/983] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:16.798235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.817044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 490/984] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:16.839738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.896826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.906017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086a944-759e-4e42-b0cc-c9c9cda554a4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.910531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:16.945166 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.945552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafdb00-d30e-4f07-9341-b0a36c68592d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:16.945775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.946068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.946312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.947123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:16.947429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.949669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:16.963576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 495/985] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:16.969770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:16.983596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:16.984126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:16.984406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:16.984679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:16.985377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:16.986745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:17.056498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 491/986] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:16 2022] POST /identity/v3/services => generated 242 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:08:17.078785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad4c08-41e8-402c-b2f9-fecac198a5f0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:17.124708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:17.125390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:17.125663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:17.125927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:17.133750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:17.135224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:17.154133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:17.155596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:17.161579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 496/987] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:17.163498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:17.273946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a32d84-7502-48b1-a5c3-21b01965f14c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:17.300839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:17.309600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:17.310143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:17.310431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:17.310850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:17.311733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:17.312096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:17.313567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773ae61-7d8b-4618-a0c7-161ccc104540 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:17.325727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 492/988] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:17.332292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:17.344761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:17.438865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9eaca-95e0-4f13-8a36-8affee549b91 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:17.472668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:17.473297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:17.473578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:17.473853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:17.474569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] RBAC: Adding query filter params (role.id=c9b6e50662504ff9922f4831d6cf4490, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=d0559bf2dae143468a9d25c82770a337) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:17.474874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:17.476280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c21017-8c7c-4888-939b-782d79e53f29 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:17.489022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 497/989] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:17 2022] GET /identity/v3/role_assignments?user.id=d0559bf2dae143468a9d25c82770a337&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=c9b6e50662504ff9922f4831d6cf4490 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:18.644866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc342e-6333-4b40-8b7c-74b4d2e733a3 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:18.645453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc342e-6333-4b40-8b7c-74b4d2e733a3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:18.645453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc342e-6333-4b40-8b7c-74b4d2e733a3 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:18.647092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 493/990] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:18.658208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:18.658607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:18.658780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:18.677326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:18.678499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:18.751048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 498/991] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:18.774049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc342e-6333-4b40-8b7c-74b4d2e733a3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:18.878359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc342e-6333-4b40-8b7c-74b4d2e733a3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:18.912728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:18.913310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:18.913589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:18.913859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:18.914438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:18.915822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:18.925363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:18.926362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:08:18.927187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:18.934432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 494/992] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 10 13:08:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:18.934566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:18.934566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.029818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9bf54-11d5-4a70-91e4-27a2801752d2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.063365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.063968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.064245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.064514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.065167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:19.065470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.066824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9ab74-f30b-430d-86ee-59c19d535127 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.076745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 499/993] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:19.077956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.103236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df5c7a-7c34-41ea-afc9-cff6b8ee4e7f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.103535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df5c7a-7c34-41ea-afc9-cff6b8ee4e7f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.103811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df5c7a-7c34-41ea-afc9-cff6b8ee4e7f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.105365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 500/994] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:19.111277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.111558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.111830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.130796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:19.131969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:19.136549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.190574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c6ffe-3292-43ca-8a44-e90cc995c1d4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.214727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.225041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 501/995] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:19.231400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.232011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.232287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.232616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.233287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:19.233589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.234940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.244694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 495/996] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:19.245098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.246002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.349542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0da9f-7ea9-4bf9-8523-c05db25f1674 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.352768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99004efc-9c67-4b48-a675-6bb1f9eb95bd None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.367610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.384318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.384951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.385256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.385528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.386105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.387459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.388883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60601ba5-5c9a-49c7-93d4-cfdb44c811ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.389662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60601ba5-5c9a-49c7-93d4-cfdb44c811ae None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.390064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60601ba5-5c9a-49c7-93d4-cfdb44c811ae None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.390340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60601ba5-5c9a-49c7-93d4-cfdb44c811ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.391032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60601ba5-5c9a-49c7-93d4-cfdb44c811ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.392695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60601ba5-5c9a-49c7-93d4-cfdb44c811ae None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:19.396729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:19.397721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:19.398593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 502/997] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:19.405308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.462601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.502951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e76033-485b-42a2-843d-5e1bc598cc4c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.536760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.537392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.537673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.537946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.538574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:19.538880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.540288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf803d-fcb7-46e0-b29a-f9614db470f4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.545478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 503/998] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:19.546938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b5eb4-630b-46cd-8645-a6f8f3ddc22e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.547220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b5eb4-630b-46cd-8645-a6f8f3ddc22e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.547495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b5eb4-630b-46cd-8645-a6f8f3ddc22e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.548688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 504/999] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:19.551368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b5eb4-630b-46cd-8645-a6f8f3ddc22e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.557397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 496/1000] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:08:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:19.558477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.558765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.559062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.579191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:19.580424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:19.669579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 497/1001] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:19.684813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b5eb4-630b-46cd-8645-a6f8f3ddc22e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.691074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.712559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.728558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.728558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.728558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.728558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.728905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] RBAC: Adding query filter params (type=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:19.728905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.730792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.736291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 505/1002] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:19.742957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.811829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.815195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765aa99-4371-486d-b992-bfd61c425470 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.849527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ac6a-53f5-4335-be8f-5b2867ce0dde None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:19.851603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.853045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.853331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.853602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] PATH_INFO: `/v3/services/image` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.854183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.855550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:19.865158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:19.866172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:19.867004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 498/1003] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:19.872880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.889758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:19.890371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:19.890650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:19.890920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:19.891622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] RBAC: Adding query filter params (interface=public, service_id=0c48df540f7742339c7eaeb2729b4d78, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:19.891927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:19.893306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:19.898458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 506/1004] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/endpoints?service_id=0c48df540f7742339c7eaeb2729b4d78&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:19.904176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:19.958555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:19.985888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d10c2-cf94-4835-baa2-a5ea0037b740 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:20.011200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fafe8f-f9b7-4c5a-bb1e-f5223af2c6e0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:20.022396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:20.023009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:20.023286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:20.023554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:20.024234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] RBAC: Adding query filter params (name=image) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:20.024542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:20.025897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:20.031745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 499/1005] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:20.037253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:20.051497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaf6dc-1dee-4c86-9e9b-ed97f0d9f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:20.052054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf6dc-1dee-4c86-9e9b-ed97f0d9f895 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:20.052334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf6dc-1dee-4c86-9e9b-ed97f0d9f895 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:20.052604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf6dc-1dee-4c86-9e9b-ed97f0d9f895 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:20.053287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaf6dc-1dee-4c86-9e9b-ed97f0d9f895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:20.054580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaf6dc-1dee-4c86-9e9b-ed97f0d9f895 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:20.062008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 507/1006] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:08:19 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:08:20.069890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:20.151639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333888a-fc77-4539-ae2a-d094b7f1c198 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:20.188049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:20.188683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:20.188966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:20.189272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:20.189925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] RBAC: Adding query filter params (type=image) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:20.190233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:20.191630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2657779-4b47-49a8-a28c-93bc1b1e94af None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:20.205308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 500/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:21.599547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad388-41a6-47e5-a21b-a9e2d113d2c5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:21.600068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad388-41a6-47e5-a21b-a9e2d113d2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:21.600068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad388-41a6-47e5-a21b-a9e2d113d2c5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:21.601600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 508/1008] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:21.606421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:21.606801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:21.606976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:21.625444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:21.626616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:21.706111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 501/1009] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:21.728893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad388-41a6-47e5-a21b-a9e2d113d2c5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:21.831821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad388-41a6-47e5-a21b-a9e2d113d2c5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:21.848051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:21.866416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b4ac6-f41b-4c92-8a22-16dba4a058b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:21.866973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4ac6-f41b-4c92-8a22-16dba4a058b6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:21.867250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4ac6-f41b-4c92-8a22-16dba4a058b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:21.867521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4ac6-f41b-4c92-8a22-16dba4a058b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:21.868629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b4ac6-f41b-4c92-8a22-16dba4a058b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:21.870082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b4ac6-f41b-4c92-8a22-16dba4a058b6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:21.872780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 509/1010] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:21.877708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:21.902846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:21.981950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37c525-2497-4d00-9a76-b09d7a7116b4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:22.017058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:22.017672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.017950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.018217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.018851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:22.020269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:22.096668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8f998-be5f-4031-859c-aa29125ecc19 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.097076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8f998-be5f-4031-859c-aa29125ecc19 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.097421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8f998-be5f-4031-859c-aa29125ecc19 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.098850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 510/1011] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:22.103684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acc34f-9155-4f11-b78d-248143a944f5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.103969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acc34f-9155-4f11-b78d-248143a944f5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.104246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acc34f-9155-4f11-b78d-248143a944f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.122654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-79acc34f-9155-4f11-b78d-248143a944f5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:22.123884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-79acc34f-9155-4f11-b78d-248143a944f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:22.137996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 502/1012] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:21 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:08:22.208054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 511/1013] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:22.230608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:22.263727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78413-fe30-428d-899a-55eb0d7d789d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.264070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78413-fe30-428d-899a-55eb0d7d789d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.264412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78413-fe30-428d-899a-55eb0d7d789d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.265945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 512/1014] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:22.273252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.273547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.273808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.294467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:22.296275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:22.319076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:22.346720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37d808-c0b4-473d-99ea-2e680b72875f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:22.376752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 513/1015] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:22.385033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:22.385652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.385931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.386198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.386776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:22.388126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:22.396781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:22.398500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:22.399672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 503/1016] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:22.400476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:22.404078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:22.448203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:22.505788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b9880-cfd4-45a1-8e65-1044e3bebb97 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:22.512663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa063-799e-4eca-ae63-16f0883a9cbf None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:22.543358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:22.544075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.544396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.545144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.546202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:22.547571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:22.550645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:22.551369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.551752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.552111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.552814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:22.553239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:22.554649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:22.559559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 504/1017] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:22.565586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:22.568203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:22.619775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 514/1018] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:22 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:22.683486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f17f-5e4b-4812-b9aa-ae97e165e5d8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:22.728952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:22.729598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.729876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.730139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.730769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:22.731065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:22.733142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbc9c0-4df8-4e05-8c10-343aef61eb21 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:22.738965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 505/1019] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 10 13:08:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:22.744668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:22.858926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dedb5-2550-4a81-835f-3e296ed440b6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:22.862388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:22.893809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeb716-e88d-484a-8729-96b988450706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:22.894430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeb716-e88d-484a-8729-96b988450706 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:22.894713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeb716-e88d-484a-8729-96b988450706 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:22.894989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeb716-e88d-484a-8729-96b988450706 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:22.895661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdeb716-e88d-484a-8729-96b988450706 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:22.897045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdeb716-e88d-484a-8729-96b988450706 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:22.971598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 515/1020] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:08:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:08:24.221861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137845d-a247-4c79-b2e2-29815639a746 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.222408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137845d-a247-4c79-b2e2-29815639a746 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.222408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137845d-a247-4c79-b2e2-29815639a746 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.223803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 506/1021] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:24.228925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.229445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.229524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.248090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:24.249299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:24.325556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 516/1022] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:24.350251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137845d-a247-4c79-b2e2-29815639a746 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:24.412255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:24.451765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137845d-a247-4c79-b2e2-29815639a746 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:24.485976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e791c-ce02-409b-9667-3b9d4e389276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:24.486531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e791c-ce02-409b-9667-3b9d4e389276 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.486900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e791c-ce02-409b-9667-3b9d4e389276 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.489324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e791c-ce02-409b-9667-3b9d4e389276 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.492417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e791c-ce02-409b-9667-3b9d4e389276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:24.493881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e791c-ce02-409b-9667-3b9d4e389276 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:24.500005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 507/1023] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:24.502367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:24.594507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:24.609706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec56dc-46d0-4d1f-95ef-3cbb86b1b82c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:24.643501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:24.644115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.644396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.644668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.645375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:24.646818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:24.678085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e17d6-f926-468f-bbcf-15fbf5aa4d40 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.678498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e17d6-f926-468f-bbcf-15fbf5aa4d40 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.678846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e17d6-f926-468f-bbcf-15fbf5aa4d40 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.680314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 508/1024] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:24.685108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.685504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.685839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.703652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:24.704830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:24.762243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 517/1025] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:08:24 2022] POST /identity/v3/users => generated 285 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:24.775915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 509/1026] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:24.800755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:24.913881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360ce43-2b10-4138-a5b5-f89ba4800ba9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:24.952258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:24.952895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:24.953829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:24.953829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] PATH_INFO: `/v3/services/image` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:24.954019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:24.955377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:24.964351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:24.965406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:24.966274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 518/1027] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Aug 10 13:08:24 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:24.972228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.033813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:25.075415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9d37f-54a3-4c50-98cc-430b328cf1b1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.109684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.110291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.110570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.110839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.111464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] RBAC: Adding query filter params (name=image) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:25.111762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.113180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3e073-aa24-412e-a6dd-bea6d988f590 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.118598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 510/1028] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:25.125322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.128506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d59fd-b991-44ec-899d-17142a106f4c None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.128799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d59fd-b991-44ec-899d-17142a106f4c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.129101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d59fd-b991-44ec-899d-17142a106f4c None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.130585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 511/1029] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:25.135432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.135723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.135873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:25.136010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.156610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:25.157827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:25.238813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 512/1030] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:25.241464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd339bed-2c2d-4a05-83c6-422bdaa8c76a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.262321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.278880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.279212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.279494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.279735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.280356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] RBAC: Adding query filter params (type=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:25.280659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.282052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.287381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 519/1031] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:25.292862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.324495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:25.365192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:25.374337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fac282-a71f-4006-bd5b-1a67394a3763 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.396124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e076e4-5f58-409a-9a6e-1c9472826785 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.414241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.414949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.415239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.415516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.428600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:25.433680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:25.434620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:25.435574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 513/1032] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:25.437810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.438424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.438706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.438978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.439680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] RBAC: Adding query filter params (interface=public, service_id=cca80f96b39b4ee297d3c91a9e7c06f4, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:25.440311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.440872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.441749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.452361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 520/1033] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/endpoints?service_id=cca80f96b39b4ee297d3c91a9e7c06f4&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:25.453604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.553226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf169e-ad29-4fbd-87c8-8798f4d09442 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.559658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec4a2-7e10-4dba-823b-47c554d72ed2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.570158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:25.593224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.593843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.594126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.594403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.595147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:25.595455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.596816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.598749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.599360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.599641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.599915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.600566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.601954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.603157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:25.617017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 521/1034] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:08:25.617230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 514/1035] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:25.620492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.731236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ade557-bc42-4b0d-a82b-39bc3ec800ae None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.770477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.771106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.771383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.771650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.783156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.784584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:25.799065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:25.800052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:25.800950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:25.800950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:25.800950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:25.800950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 522/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:25.806173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:25.911477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77187b-38c3-4300-a3bc-33232b0e923e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:25.946238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:25.946855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:25.947139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:25.947414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:25.948065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:25.948370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:25.949832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:25.963561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 515/1037] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:25.970210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:26.021395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:26.072247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec780cb-5204-4a9f-9a30-1191207a8720 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:26.106668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:26.107537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.107893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.108270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.115754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:26.117202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:26.128981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:26.129976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:08:26.130917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:26.132118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 523/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:08:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:26.136071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:26.171618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:26.235904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9d246-40a6-4558-a144-38ff3230bbff None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:26.270644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:26.271259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.271524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.271795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.272522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:26.272827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:26.274304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:26.291793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 516/1039] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:08:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:26.298858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:26.400492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49192d4-7bbc-4459-a413-043d135ca5cd None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:26.411638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:26.436899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:26.437544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.437827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.438101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.507130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:26.508572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:26.606475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 524/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:26.611997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:26.717800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:26.724379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2185e-8c3f-413f-8f90-149aa5ee569e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:26.761483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:26.762122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.762404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.762675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.776275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:26.778015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-c45faec5-1d85-4d0f-858e-0c9217d78c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:26.793365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:26.794338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:26.800972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:26.800972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:26.800972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:26.800972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 517/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:26.801755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:26.815211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:26.859305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6b28-15b1-4f5a-ac84-c2f144714019 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.859591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6b28-15b1-4f5a-ac84-c2f144714019 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.859870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6b28-15b1-4f5a-ac84-c2f144714019 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.861232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 518/1042] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:26.866055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e844b-96f6-4b73-b20c-04051037417c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.866334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e844b-96f6-4b73-b20c-04051037417c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.866603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e844b-96f6-4b73-b20c-04051037417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.884511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-182e844b-96f6-4b73-b20c-04051037417c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:26.885738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-182e844b-96f6-4b73-b20c-04051037417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:26.912846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f305c-edba-4a47-a1b5-653a7ccc4e7e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:26.953487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:26.954093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:26.954370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:26.954638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:26.955345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:26.955648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:26.957018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:26.961118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 519/1043] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:26.969162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 525/1044] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:26.974477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e844b-96f6-4b73-b20c-04051037417c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:26.981067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.054223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.055077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.077931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e844b-96f6-4b73-b20c-04051037417c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.089266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4e16-c4dc-4308-8f02-eed45922d5ae None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.114871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.115509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.115877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.116141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.123583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.125078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.127039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.127859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.128278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.128689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:27.136438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:27.137449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:27.138825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 520/1045] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:27.142195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:27.143585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:27.148022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:27.149133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:27.149983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:27.153123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 526/1046] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:27.156464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.254561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c08cb-9db6-464c-a71c-e16860a99a5d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.266046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9c2bb-9bdc-4803-9469-e3f5830aee7b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.292563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.293251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.293518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.293733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.294424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:27.294767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.296184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.297987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.299383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.304340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.304947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.305314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.314146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.314146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:27.314146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.314146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.314474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 521/1047] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:27.315224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.316438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 527/1048] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:27.321949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.420915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5974b-8127-4ca7-a895-52f7ec8ff371 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.430207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af517c3-533d-488c-9431-0e78e1ed30bd None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.457576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.458191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.458466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.458738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.459459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] RBAC: Adding query filter params (role.id=c9b6e50662504ff9922f4831d6cf4490, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=5dda95d65ff54916b0565d412c78e75f) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:27.459773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.461214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.466134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12664a1b-7289-4804-a74e-4dece1f751e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.467910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 522/1049] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/role_assignments?user.id=5dda95d65ff54916b0565d412c78e75f&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=c9b6e50662504ff9922f4831d6cf4490 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:27.468168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12664a1b-7289-4804-a74e-4dece1f751e1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.468168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12664a1b-7289-4804-a74e-4dece1f751e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.468168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12664a1b-7289-4804-a74e-4dece1f751e1 None admin] PATH_INFO: `/v3/users/1198d57c0ca94b08bf5bfb329c3f83e0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.483967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12664a1b-7289-4804-a74e-4dece1f751e1 None admin] RBAC: Authorizing `identity:get_user(user_id=1198d57c0ca94b08bf5bfb329c3f83e0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.485482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12664a1b-7289-4804-a74e-4dece1f751e1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.487468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 528/1050] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/users/1198d57c0ca94b08bf5bfb329c3f83e0 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:27.493252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.559320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.604105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21636c-6b9a-478a-a295-4a5aa3b982d3 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.645551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.646556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.647062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.647543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.675937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01ef9-d3a1-445c-afdc-3d63cbebb782 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.676431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01ef9-d3a1-445c-afdc-3d63cbebb782 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.676847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01ef9-d3a1-445c-afdc-3d63cbebb782 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.678391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 529/1051] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:27.683371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.684174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.684672 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.704803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:27.706145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:27.723080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.724545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.755744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.777050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 530/1052] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:27.797552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:27.800665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.849116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 523/1053] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:27.850706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.918688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec44598-4c4b-4126-b58a-118525f27ddc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.957120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c2b7-3807-4743-9026-38667cbdb9b8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:27.958810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.960236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.960524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.960884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] PATH_INFO: `/v3/services/image` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.961470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.962834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:27.974200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:27.975231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:27.981337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 531/1054] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:27.982692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:27.991484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:27.992118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:27.992397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:27.992681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:27.993849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] RBAC: Authorizing `identity:get_project(project_id=0d838b85d5144f2898444814b8efb2a0)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:27.995276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:27.997236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 524/1055] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:28.003453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:28.033468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:28.035488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:28.089881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec12e0c-7257-4bf8-ab10-6203a131c57b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:28.109711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c255afe-0ce9-438e-84ac-8d0cecd8b644 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:28.124455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:28.125108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:28.125394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:28.125671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:28.126304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] RBAC: Adding query filter params (name=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:28.126613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:28.127977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:28.132887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 532/1056] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:28.139221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:28.153809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:28.154478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:28.154757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:28.155025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:28.155926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=0d838b85d5144f2898444814b8efb2a0, user.id=1198d57c0ca94b08bf5bfb329c3f83e0) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:28.156355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:28.157886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:28.173030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 525/1057] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:28 2022] GET /identity/v3/role_assignments?user.id=1198d57c0ca94b08bf5bfb329c3f83e0&scope.project.id=0d838b85d5144f2898444814b8efb2a0&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:28.257856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14c33-2fce-43e9-aa21-e252a50b7d21 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:28.267942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:28.295960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:28.297104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:28.297386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:28.297659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:28.298299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:28.298604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:28.299989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73c8e1-2069-4998-9287-083466aaada9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:28.305734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 533/1058] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:28.311831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:28.406335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:28.415147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3841b-28b6-4629-b938-b58594fead9d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:28.449036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5cc69-92f0-41ce-8442-72c16dbab682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:28.450383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5cc69-92f0-41ce-8442-72c16dbab682 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:28.450683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5cc69-92f0-41ce-8442-72c16dbab682 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:28.451073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5cc69-92f0-41ce-8442-72c16dbab682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:28.451771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5cc69-92f0-41ce-8442-72c16dbab682 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:28.453153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5cc69-92f0-41ce-8442-72c16dbab682 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:28.537321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 526/1059] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:08:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:29.477612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c39e8-51ca-407d-a433-78ecd4e0cab3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:29.478161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c39e8-51ca-407d-a433-78ecd4e0cab3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:29.478161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c39e8-51ca-407d-a433-78ecd4e0cab3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:29.479505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 534/1060] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:29.485490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:29.485903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:29.486112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:29.504556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:29.505757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:29.580959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 527/1061] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:29.601859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c39e8-51ca-407d-a433-78ecd4e0cab3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:29.717233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c39e8-51ca-407d-a433-78ecd4e0cab3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:29.753073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:29.754031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:29.754522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:29.755031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:29.766661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:29.771626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:29.772572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:29.773445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:29.774614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 535/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 10 13:08:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:29.781090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:29.856560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:29.882164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70100bf-e70f-4efc-a192-1dce31bde6da None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:29.925724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:29.926490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:29.926889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:29.927309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:29.928096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:29.928516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:29.930056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:29.944383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 528/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:08:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:29.945702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:29.979248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.048705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096bbf-94ed-4ada-9027-a43b2e882b69 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.082467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.083116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.083380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.083650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.095490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.096928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-267ce648-af62-4f68-8613-ec6990acc711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:30.114829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:30.115905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:30.116809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:30.116809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:30.116809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:30.116809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 536/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:30.123859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.189071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56e8b-086c-4df0-a50e-12d604cbfc65 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.189384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56e8b-086c-4df0-a50e-12d604cbfc65 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.189626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56e8b-086c-4df0-a50e-12d604cbfc65 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.191103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 537/1065] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:30.195877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.196167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.196442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.214286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:30.215486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:30.235929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87583a83-7531-4950-bdc2-2829d2df5327 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.279038 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.279759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.280059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.280317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.280964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:30.281298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.282664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.285364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.285487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 538/1066] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:30.300803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 529/1067] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:30.301915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.308375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.318888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.406257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79cfc9-5c3b-47a7-873c-3b2422ab21f9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.412724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51099b50-f28a-428c-8b58-8ea6af300649 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.441630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.442266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.442628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.442815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.448640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.449304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.449587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.449862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.450650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.452188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.460360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:30.463802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:30.464721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:08:30.471491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:30.472113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:30.472957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:30.473835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:30.474471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 539/1068] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:30.474541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 530/1069] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:30.474608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.475709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.535703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.562779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.583230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2686cd-c0d9-43bc-97e7-92e25c173594 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.588081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2fe1-fdac-484f-9c90-c0986dc85abf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.618698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.619295 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.619574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.619841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.620496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:30.620872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.622265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa38de-ee75-4606-a14e-95c0e5ff99bd None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.623523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.624312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.624598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.624865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.625641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:30.625946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.627415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64704a5e-df91-451b-95b8-1d39495d2850 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.631190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 531/1070] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:30.632339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec9678-30cd-4084-86a5-f521c24b1005 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.632624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec9678-30cd-4084-86a5-f521c24b1005 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.632896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec9678-30cd-4084-86a5-f521c24b1005 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.634284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 532/1071] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:30.637254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec9678-30cd-4084-86a5-f521c24b1005 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.646605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 540/1072] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:30.651644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.651931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.652204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.671463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:30.672647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:30.737016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 541/1073] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:30.737874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.745329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec9678-30cd-4084-86a5-f521c24b1005 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.765777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.783740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.784374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.784655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.784925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] PATH_INFO: `/v3/users/1198d57c0ca94b08bf5bfb329c3f83e0` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.786080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] RBAC: Authorizing `identity:get_user(user_id=1198d57c0ca94b08bf5bfb329c3f83e0)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.787504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.793788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 533/1074] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/users/1198d57c0ca94b08bf5bfb329c3f83e0 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:30.793788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.803949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.842553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324c5be-876b-420a-80bc-9c67e6c05cb5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.875278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.875913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.876191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.876459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.886218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:30.887651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:30.891606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37d62a-67fc-4e67-8f29-2077c556de02 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:30.901970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:30.903202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:30.904768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:30.904768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:30.904768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:30.904768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 542/1075] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:30.904768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:30.930496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:30.931110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:30.931428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:30.931698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:30.986165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:30.999181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.000636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.002184 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4f1d7-5eba-4add-ac1b-0c3678dc5a22 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.016385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:31.036415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.037062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.037341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.037610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.038705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] RBAC: Authorizing `identity:get_project(project_id=0d838b85d5144f2898444814b8efb2a0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.040100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.042455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 543/1076] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:31.043371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.100780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 534/1077] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:31.101550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.147229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2125-8125-4681-934f-c017ffe03ece None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.181299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.181901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.182181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.182453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.183082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:31.183382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.184780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.185535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:31.196642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 544/1078] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:31.197465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.199980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd0c8-baf2-4e3a-a175-ad3341c5bcce None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.236264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.236906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.237213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.237484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0/users/1198d57c0ca94b08bf5bfb329c3f83e0/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.239345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d838b85d5144f2898444814b8efb2a0, user_id=1198d57c0ca94b08bf5bfb329c3f83e0, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.240789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.242682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:31.298627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e2d6b-7ff9-4582-a936-3cee69af93f7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.299560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 535/1079] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:08:31 2022] PUT /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0/users/1198d57c0ca94b08bf5bfb329c3f83e0/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:08:31.300373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.343303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.344451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.344935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.345522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.358313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.360103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:31.378076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:31.379131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:31.380066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:31.380066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:31.380066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:31.386950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 545/1080] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:31.388752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.430117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76bf1c-ad0b-4b3d-b545-f5c5116015a8 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.500530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.501221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.501482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.501758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.508431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.509898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:31.521551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:31.522648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:31.523574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 536/1081] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:31.528602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.537549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:31.540684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:31.544075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5731b4-cc13-4aa7-ba33-3b41248ff463 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.580119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.580744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.581020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.581302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.581947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:31.582262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.583599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.596791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 546/1082] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:31.643833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da68e78-e908-4ea2-b987-1d9658098fab None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.685909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.686536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.686821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.687097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.687807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:31.688118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.689631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3be433-966b-43a8-bc3c-14f31fd67ab8 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.706677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 537/1083] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:31.712582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:31.821813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624be6a-b50b-4500-9063-54441b3b7904 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:31.864354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4292fd8-ac5a-4124-a648-4f68e9559a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:31.865060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4292fd8-ac5a-4124-a648-4f68e9559a3e None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:31.865348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4292fd8-ac5a-4124-a648-4f68e9559a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:31.865619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4292fd8-ac5a-4124-a648-4f68e9559a3e None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/5dda95d65ff54916b0565d412c78e75f/roles/c9b6e50662504ff9922f4831d6cf4490` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:31.867768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4292fd8-ac5a-4124-a648-4f68e9559a3e None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=5dda95d65ff54916b0565d412c78e75f, role_id=c9b6e50662504ff9922f4831d6cf4490)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:31.869226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4292fd8-ac5a-4124-a648-4f68e9559a3e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:31.963832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 547/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:08:31 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/5dda95d65ff54916b0565d412c78e75f/roles/c9b6e50662504ff9922f4831d6cf4490 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:08:33.351272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b725be0-941c-4ae8-8ce2-4fe08538a930 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.351821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b725be0-941c-4ae8-8ce2-4fe08538a930 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.351821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b725be0-941c-4ae8-8ce2-4fe08538a930 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.353199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 538/1085] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:33.359041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.359434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.359596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.377839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:33.379031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:33.388148 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:33.445040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 548/1086] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:33.468285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b725be0-941c-4ae8-8ce2-4fe08538a930 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:33.533939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:33.568726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b725be0-941c-4ae8-8ce2-4fe08538a930 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:33.602838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:33.603388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.603668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.603938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.613833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:33.618616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:33.619593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:33.620426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 539/1087] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:08:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:33.626278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:33.725893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5659861-dae2-4d15-8730-9f049fdae5c8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:33.759167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:33.759770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.760052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.760325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.760983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:33.761311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:33.762658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:33.775681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 549/1088] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:33.776811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:33.879208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db8a17-d49b-4ddd-8f03-f549001ed0df None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:33.908131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:33.916528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870b0f-7fad-4d44-a9b6-6e69494b7934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:33.917192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870b0f-7fad-4d44-a9b6-6e69494b7934 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.917476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870b0f-7fad-4d44-a9b6-6e69494b7934 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.918016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870b0f-7fad-4d44-a9b6-6e69494b7934 None admin] PATH_INFO: `/v3/users/1198d57c0ca94b08bf5bfb329c3f83e0` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.918857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870b0f-7fad-4d44-a9b6-6e69494b7934 None admin] RBAC: Authorizing `identity:get_user(user_id=1198d57c0ca94b08bf5bfb329c3f83e0)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:33.920366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870b0f-7fad-4d44-a9b6-6e69494b7934 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:33.921999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 540/1089] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:33 2022] GET /identity/v3/users/1198d57c0ca94b08bf5bfb329c3f83e0 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:33.927692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:33.937152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:33.951990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cb871-e1a2-482c-9dbd-2721da908f14 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.952280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cb871-e1a2-482c-9dbd-2721da908f14 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.952549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cb871-e1a2-482c-9dbd-2721da908f14 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.953930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 541/1090] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:08:33.958861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:33.959138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:33.959413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:33.977759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:33.978962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:34.027306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e69c44-fd6a-4152-a3b4-4865a318bc7c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.044800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 542/1091] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:08:34.060563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.061200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.061484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.061760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.067490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.130562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.131941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.140206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.165617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8700-f785-4b5f-b6df-39229da95496 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.190585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.202655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.203294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.203575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.203849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.213683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:34.219145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:34.220068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:34.221170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 543/1092] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:34.226182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.240589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 550/1093] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:34.241628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.325460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df89122-c362-454a-959e-523a5b2fdc2e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.336768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001677-4919-4adf-bad6-d52535aa8652 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.344593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.360715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.363057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.363057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.363057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.363057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:34.363057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.364129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.373042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 544/1094] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:34.373184 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.373663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.373754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.373998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.375107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] RBAC: Authorizing `identity:get_project(project_id=0d838b85d5144f2898444814b8efb2a0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.376522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.377898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.378913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 551/1095] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:34.384231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.424301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.478696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a947395-c860-4a35-8846-5d6542a7565e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.481727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6bd76-fbae-4654-a4ff-9244a83dc06e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.512022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.512647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.512929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.513252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.515911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.516518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.516793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.517090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.517810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=0d838b85d5144f2898444814b8efb2a0, user.id=1198d57c0ca94b08bf5bfb329c3f83e0) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:34.518183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.519624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.524158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.525478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 552/1096] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/role_assignments?user.id=1198d57c0ca94b08bf5bfb329c3f83e0&scope.project.id=0d838b85d5144f2898444814b8efb2a0&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:08:34.525623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:34.540351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:34.541465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:34.542388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:34.542388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:34.542388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:34.542388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 545/1097] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:34.548258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.590044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.656216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e1e49-9ff4-4db0-a63e-13fbd36660b2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.694518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296389-c240-494f-9e77-153a811508fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.695135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296389-c240-494f-9e77-153a811508fa None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.695414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296389-c240-494f-9e77-153a811508fa None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.695688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296389-c240-494f-9e77-153a811508fa None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.696337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296389-c240-494f-9e77-153a811508fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:34.696645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296389-c240-494f-9e77-153a811508fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.698035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296389-c240-494f-9e77-153a811508fa None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.709892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 553/1098] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:34.715748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.811387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.815444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51cc86-b9c6-4050-9516-f73ad54509d0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:34.849207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:34.849842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:34.850124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:34.850396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:34.858595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:34.860020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:08:34.870930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:34.871860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:34.873499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 546/1099] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:34.878294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296389-c240-494f-9e77-153a811508fa None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:34.974435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:34.978218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296389-c240-494f-9e77-153a811508fa None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:35.011532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:35.012134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:35.012412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:35.012682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:35.013596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:35.014078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:35.015500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:35.034084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 554/1100] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 10 13:08:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:35.039726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:35.139613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c5b9d-9e63-4862-8bde-732106834657 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:35.172018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:35.172620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:35.172902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:35.173202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:35.199626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:35.239239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:35.240580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:35.339278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 547/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:35.345794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:35.441152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9d09e-adab-42ba-b5be-f6b280c0a4ef None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:35.473890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:35.474504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:35.474782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:35.475055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:35.485618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:35.487034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:08:35.501865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:35.502986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:08:35.504124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:08:35.504124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:08:35.504124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:35.504124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 555/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:35.510285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:35.566864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:35.608145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aadaf4-f5c1-4781-9445-1c5246fd789d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:35.641772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:35.642376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:35.642653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:35.642920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:35.643560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:35.643857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:35.645249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:35.658665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 548/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:35.664540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:35.677252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:35.764470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553e28-7661-46b2-87fa-4d7e05d863f1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:35.795881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:35.796496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:35.796771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:35.797062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:35.803411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:35.804832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:08:35.816011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:35.817103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:08:35.818055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:35.821129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 556/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:08:35.822833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:35.920796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5f8e7-ab21-44ef-b74f-97413d52d98d None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:35.937578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:35.954292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:35.955042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:35.955212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:35.955524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:35.956217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:35.956524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:35.958004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cf43-9c60-4148-8cca-20b0b986ee26 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:35.970239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 549/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:35.975763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:36.039500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:36.072633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7862c1-58d4-43cf-9b19-6da41a65b5ae None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:36.104257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:36.104849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:36.105156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:36.105427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:36.106139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] RBAC: Adding query filter params (role.id=c9b6e50662504ff9922f4831d6cf4490, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=5dda95d65ff54916b0565d412c78e75f) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:36.106443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:36.108413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d2baa-ae58-448c-b545-fea895ce2016 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:36.114057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 557/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:35 2022] GET /identity/v3/role_assignments?user.id=5dda95d65ff54916b0565d412c78e75f&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=c9b6e50662504ff9922f4831d6cf4490 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:36.582693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a17ced-91d9-45d4-8636-480c07d04456 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:36.583205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a17ced-91d9-45d4-8636-480c07d04456 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:36.583316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a17ced-91d9-45d4-8636-480c07d04456 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:36.584585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 550/1107] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:36.589901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:36.590304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:36.590449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:36.609579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:36.610981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:36.680821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 558/1108] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:36.705616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a17ced-91d9-45d4-8636-480c07d04456 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:36.810275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a17ced-91d9-45d4-8636-480c07d04456 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:36.846806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8fbf4-82c4-4d51-b54e-b4d540df80a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:36.847383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8fbf4-82c4-4d51-b54e-b4d540df80a6 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:36.847662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8fbf4-82c4-4d51-b54e-b4d540df80a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:36.847931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8fbf4-82c4-4d51-b54e-b4d540df80a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:36.849077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8fbf4-82c4-4d51-b54e-b4d540df80a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:36.850513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8fbf4-82c4-4d51-b54e-b4d540df80a6 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:36.852084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 551/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:36.859191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:36.920907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:36.960295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f26c78-4627-4184-8496-320bf3d05c23 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:36.994406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96efe6b-952a-4f70-9c4e-05ec2ed26502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:36.995003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96efe6b-952a-4f70-9c4e-05ec2ed26502 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:36.995282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96efe6b-952a-4f70-9c4e-05ec2ed26502 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:36.995550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96efe6b-952a-4f70-9c4e-05ec2ed26502 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:36.996195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96efe6b-952a-4f70-9c4e-05ec2ed26502 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:36.998406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96efe6b-952a-4f70-9c4e-05ec2ed26502 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:37.089075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 559/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:08:36 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:39.092660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f47b-3f83-411a-8560-017c1f26c3f0 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:39.093243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f47b-3f83-411a-8560-017c1f26c3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:39.093243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f47b-3f83-411a-8560-017c1f26c3f0 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:39.094567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 552/1111] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:39.099711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:39.100103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:39.100269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:39.118617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:39.119805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:39.184873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 560/1112] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:39.208258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f47b-3f83-411a-8560-017c1f26c3f0 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:39.255891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:39.309218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f47b-3f83-411a-8560-017c1f26c3f0 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:39.343340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:39.343896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:39.344171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:39.344440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] PATH_INFO: `/v3/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:39.345825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] RBAC: Authorizing `identity:get_role(role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:39.347177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:39.348485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 553/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:39 2022] GET /identity/v3/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:39.354375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:39.402041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:39.450288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba22e-4ea3-4e0c-9965-ab92d13bdf0e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:39.484177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:39.484795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:39.485103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:39.485379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] PATH_INFO: `/v3/users/1cf4aadd348b44a49afc2fb347721756` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:39.498070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf4aadd348b44a49afc2fb347721756)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:39.499615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:39.501267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 561/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:39 2022] GET /identity/v3/users/1cf4aadd348b44a49afc2fb347721756 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:39.506997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:39.601705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f371f09-9433-42bf-bf22-b45cc5985686 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:39.615470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:39.634618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:39.635211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:39.635488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:39.635756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:39.703272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:39.704716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:39.800185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 554/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:39.806079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:39.905425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09c665-fc17-4a88-b8f0-0019ad2a5496 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:39.912411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:39.939746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd1507-1d89-45d8-aa76-b38548fb1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:39.940379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd1507-1d89-45d8-aa76-b38548fb1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:39.940653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd1507-1d89-45d8-aa76-b38548fb1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:39.940919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd1507-1d89-45d8-aa76-b38548fb1cb5 None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:39.942039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd1507-1d89-45d8-aa76-b38548fb1cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=0d838b85d5144f2898444814b8efb2a0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:39.943506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd1507-1d89-45d8-aa76-b38548fb1cb5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:39.945353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 562/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:39 2022] GET /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:39.951573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:39.966085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:40.048092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326c15-072b-4b0f-9559-c14d44186e51 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:40.081248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:40.081844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:40.082122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:40.082395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:40.083107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] RBAC: Adding query filter params (role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=0d838b85d5144f2898444814b8efb2a0, user.id=1cf4aadd348b44a49afc2fb347721756) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:40.083414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:40.084821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e7fd5-fc14-4da5-a83b-3b6c16c78030 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:40.090048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 555/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:39 2022] GET /identity/v3/role_assignments?user.id=1cf4aadd348b44a49afc2fb347721756&scope.project.id=0d838b85d5144f2898444814b8efb2a0&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:42.088780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d596f-e3e3-48ae-8b47-5f70b4ce1a65 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:42.089384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d596f-e3e3-48ae-8b47-5f70b4ce1a65 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:42.089384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d596f-e3e3-48ae-8b47-5f70b4ce1a65 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:42.090688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 563/1118] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:42.096151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:42.096654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:42.096759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:42.114619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:42.115785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:42.181480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 556/1119] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:42.204856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d596f-e3e3-48ae-8b47-5f70b4ce1a65 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:42.302252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d596f-e3e3-48ae-8b47-5f70b4ce1a65 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:42.335139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:42.335756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:42.336035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:42.336305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] PATH_INFO: `/v3/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:42.337704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] RBAC: Authorizing `identity:get_role(role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:42.339098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:42.340346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 564/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:42 2022] GET /identity/v3/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:42.346227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:42.355970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:42.441407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc80cf-8931-4f65-9cd0-d9795bc9788a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:42.473934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:42.474549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:42.474826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:42.475096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] PATH_INFO: `/v3/users/1cf4aadd348b44a49afc2fb347721756` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:42.476186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] RBAC: Authorizing `identity:get_user(user_id=1cf4aadd348b44a49afc2fb347721756)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:42.477626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:42.479173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 557/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:42 2022] GET /identity/v3/users/1cf4aadd348b44a49afc2fb347721756 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:42.485561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:42.526714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:42.580765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd02a26-8579-484e-87db-962be73995f3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:42.613607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa6994-d326-4086-bad9-5623fab02bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:42.614168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa6994-d326-4086-bad9-5623fab02bcb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:42.614444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa6994-d326-4086-bad9-5623fab02bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:42.614717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa6994-d326-4086-bad9-5623fab02bcb None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:42.615813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa6994-d326-4086-bad9-5623fab02bcb None admin] RBAC: Authorizing `identity:get_project(project_id=0d838b85d5144f2898444814b8efb2a0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:42.617239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa6994-d326-4086-bad9-5623fab02bcb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:42.618840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 565/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:42 2022] GET /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:42.624789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:42.700903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:42.723103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27e12c-4662-43f2-9319-a19b454103bb None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:42.757132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c07e4-2cc4-41ea-a01a-58e7243b9435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:42.757784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c07e4-2cc4-41ea-a01a-58e7243b9435 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:42.758066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c07e4-2cc4-41ea-a01a-58e7243b9435 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:42.758342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c07e4-2cc4-41ea-a01a-58e7243b9435 None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0/users/1cf4aadd348b44a49afc2fb347721756/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:42.760466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c07e4-2cc4-41ea-a01a-58e7243b9435 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d838b85d5144f2898444814b8efb2a0, user_id=1cf4aadd348b44a49afc2fb347721756, role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:42.761923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c07e4-2cc4-41ea-a01a-58e7243b9435 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:42.826176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 558/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:08:42 2022] PUT /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0/users/1cf4aadd348b44a49afc2fb347721756/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:08:44.772456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557942a-7a28-4692-b801-7ddea7dc6144 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:44.772986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557942a-7a28-4692-b801-7ddea7dc6144 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:44.772986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557942a-7a28-4692-b801-7ddea7dc6144 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:44.774345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 566/1124] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:44.780316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:44.780706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:44.780864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:44.799119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:44.800299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:44.865520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 559/1125] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:44.888412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557942a-7a28-4692-b801-7ddea7dc6144 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:44.987013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557942a-7a28-4692-b801-7ddea7dc6144 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:44.996982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:45.018666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:45.019276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:45.019550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:45.019818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] PATH_INFO: `/v3/roles/3efd53fc5b7044ae9256b77c77e6a918` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:45.021201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] RBAC: Authorizing `identity:get_role(role_id=3efd53fc5b7044ae9256b77c77e6a918)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:45.022537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:45.023813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 567/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:44 2022] GET /identity/v3/roles/3efd53fc5b7044ae9256b77c77e6a918 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:45.030346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:45.093680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:45.126050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef764d2-9098-4c7f-a3dc-730f5053badb None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:45.159241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:45.159864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:45.160144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:45.160415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] PATH_INFO: `/v3/users/1cf4aadd348b44a49afc2fb347721756` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:45.161545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf4aadd348b44a49afc2fb347721756)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:45.162963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:45.164519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 560/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:45 2022] GET /identity/v3/users/1cf4aadd348b44a49afc2fb347721756 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:45.170842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:45.267495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6b32a-955f-4b43-aad9-71e33b9cec86 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:45.301006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:45.301605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:45.301976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:45.302250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:45.347507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:45.368519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:45.369965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:45.464798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 568/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:45.471337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:45.572695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac3096-0a65-4cef-81d4-95fdb8af0274 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:45.606274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0b6b7-65bb-4937-a34c-9f6eec94a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:45.607612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0b6b7-65bb-4937-a34c-9f6eec94a474 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:45.607897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0b6b7-65bb-4937-a34c-9f6eec94a474 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:45.608169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0b6b7-65bb-4937-a34c-9f6eec94a474 None admin] PATH_INFO: `/v3/projects/0d838b85d5144f2898444814b8efb2a0` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:45.609307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0b6b7-65bb-4937-a34c-9f6eec94a474 None admin] RBAC: Authorizing `identity:get_project(project_id=0d838b85d5144f2898444814b8efb2a0)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:45.610721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0b6b7-65bb-4937-a34c-9f6eec94a474 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:45.612344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 561/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:45 2022] GET /identity/v3/projects/0d838b85d5144f2898444814b8efb2a0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:45.618627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:45.693785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:45.713903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de62b59-53f0-444c-a22a-477a9b9e4385 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:45.747136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:45.747797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:45.748070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:45.748338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:45.749071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] RBAC: Adding query filter params (role.id=3efd53fc5b7044ae9256b77c77e6a918, scope.project.id=0d838b85d5144f2898444814b8efb2a0, user.id=1cf4aadd348b44a49afc2fb347721756) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:45.749379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:45.750774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994d016-0e07-4a62-974f-f68ccce8b50b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:45.756312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 569/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:45 2022] GET /identity/v3/role_assignments?user.id=1cf4aadd348b44a49afc2fb347721756&scope.project.id=0d838b85d5144f2898444814b8efb2a0&role.id=3efd53fc5b7044ae9256b77c77e6a918 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:47.791595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f078-acd6-439b-8eda-2328c123e343 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:47.792120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f078-acd6-439b-8eda-2328c123e343 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:47.792120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f078-acd6-439b-8eda-2328c123e343 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:47.793515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 562/1131] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:47.799413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:47.799833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:47.799961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:47.817389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:47.818547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:47.887847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 570/1132] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:47.911575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15f078-acd6-439b-8eda-2328c123e343 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:47.914392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:48.010708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15f078-acd6-439b-8eda-2328c123e343 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:48.043453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d43145-d168-4815-bd5a-53276bf292c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:48.044065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d43145-d168-4815-bd5a-53276bf292c1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:48.045087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d43145-d168-4815-bd5a-53276bf292c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:48.045378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d43145-d168-4815-bd5a-53276bf292c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:48.046478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d43145-d168-4815-bd5a-53276bf292c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:48.047891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d43145-d168-4815-bd5a-53276bf292c1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:48.049466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 563/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:48.055133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:48.121137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:48.149927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d54a5-54d3-442a-94df-81fa1f2d84fa None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:48.189093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8448-e5ea-470f-873b-d7b3d9c1c698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:48.189632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8448-e5ea-470f-873b-d7b3d9c1c698 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:48.189913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8448-e5ea-470f-873b-d7b3d9c1c698 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:48.190186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8448-e5ea-470f-873b-d7b3d9c1c698 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:48.190834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d8448-e5ea-470f-873b-d7b3d9c1c698 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:48.192292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d8448-e5ea-470f-873b-d7b3d9c1c698 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:48.292213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 571/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:08:48 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:50.296224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8ad8e-a228-449f-b37a-711c80c779d6 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:50.296750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8ad8e-a228-449f-b37a-711c80c779d6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:50.296750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8ad8e-a228-449f-b37a-711c80c779d6 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:50.298134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 564/1135] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:50.303316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:50.303744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:50.303848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:50.321430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:50.322606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:50.386734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 572/1136] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:50.411393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8ad8e-a228-449f-b37a-711c80c779d6 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:50.479232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:50.520390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8ad8e-a228-449f-b37a-711c80c779d6 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:50.555275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d13058-e604-428c-a9c7-f37eb5e954c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:50.555830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d13058-e604-428c-a9c7-f37eb5e954c1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:50.556108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d13058-e604-428c-a9c7-f37eb5e954c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:50.556379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d13058-e604-428c-a9c7-f37eb5e954c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:50.557492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d13058-e604-428c-a9c7-f37eb5e954c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:50.558905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d13058-e604-428c-a9c7-f37eb5e954c1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:50.560986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 565/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:08:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:50.566830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:50.619313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:50.663562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429f3d8-47c7-43c3-b422-16ab02be80f3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:50.696628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9436ed0-1f1c-43ee-9937-6669afbb0b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:50.697247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9436ed0-1f1c-43ee-9937-6669afbb0b8f None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:50.697526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9436ed0-1f1c-43ee-9937-6669afbb0b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:50.697794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9436ed0-1f1c-43ee-9937-6669afbb0b8f None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:50.698428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9436ed0-1f1c-43ee-9937-6669afbb0b8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:50.699825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9436ed0-1f1c-43ee-9937-6669afbb0b8f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:50.785989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 573/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:08:50 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:08:52.838599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4825-4507-4ae1-9276-3c91deec2b30 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:52.839141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4825-4507-4ae1-9276-3c91deec2b30 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:52.839141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4825-4507-4ae1-9276-3c91deec2b30 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:52.840490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 566/1139] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:52.845950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:52.846346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:52.846499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:52.864736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:52.865951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:52.930071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 574/1140] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:52.953137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e4825-4507-4ae1-9276-3c91deec2b30 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:53.053278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e4825-4507-4ae1-9276-3c91deec2b30 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:53.085796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:53.086414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:53.086696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:53.086973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:53.094100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:53.097813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:53.102445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:53.103583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:53.104485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 567/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:08:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:53.110201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:53.149545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:53.207027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e38906-fa24-4442-89e4-1f7da46139d6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:53.241406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:53.242003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:53.242281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:53.242549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:53.243201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:53.243499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:53.244838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:53.253319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 575/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:53.259035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:53.355765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8efc3d-ec81-490b-a7ca-f5e78d7a5c4b None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:53.390557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:53.391177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:53.391457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:53.391727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] PATH_INFO: `/v3/users/bfe16a7f8f97488e812b13319e98c3d6` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:53.405327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] RBAC: Authorizing `identity:get_user(user_id=bfe16a7f8f97488e812b13319e98c3d6)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:53.406756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:53.408420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 568/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:53 2022] GET /identity/v3/users/bfe16a7f8f97488e812b13319e98c3d6 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:53.414270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:53.511543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf7dc4-3b7b-4ba0-a5b4-5eac7c8e3058 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:53.513455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:53.553133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:53.553738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:53.554015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:53.554287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:53.619191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:53.620621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:53.716610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 576/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:53.722459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:53.775983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:53.816877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a60c78-3514-4056-b502-5b31c302e59a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:53.849139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d518e-e637-4905-a0e6-b0b981d1755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:53.849754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d518e-e637-4905-a0e6-b0b981d1755f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:53.850029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d518e-e637-4905-a0e6-b0b981d1755f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:53.850295 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d518e-e637-4905-a0e6-b0b981d1755f None admin] PATH_INFO: `/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:53.851385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d518e-e637-4905-a0e6-b0b981d1755f None admin] RBAC: Authorizing `identity:get_project(project_id=1f6a02dfdd7840dda1ed7e7934c3880d)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:53.852791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d518e-e637-4905-a0e6-b0b981d1755f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:53.854415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 569/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:53 2022] GET /identity/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:53.860552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:53.862741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:53.959187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b18d2c-1798-4bd3-9235-1881998e5fd5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:53.992748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:53.993366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:53.993642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:53.993905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:53.994612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=1f6a02dfdd7840dda1ed7e7934c3880d, user.id=bfe16a7f8f97488e812b13319e98c3d6) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:53.994914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:53.996310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd477e2-6f86-48ed-8fd7-d045885e48e5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:54.004760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 577/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:53 2022] GET /identity/v3/role_assignments?user.id=bfe16a7f8f97488e812b13319e98c3d6&scope.project.id=1f6a02dfdd7840dda1ed7e7934c3880d&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:55.962578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a641fb-d5b6-4e8a-b496-4bc1c6850d7e None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:55.963088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a641fb-d5b6-4e8a-b496-4bc1c6850d7e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:55.963088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a641fb-d5b6-4e8a-b496-4bc1c6850d7e None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:55.964394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 570/1147] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:55.969961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:55.970359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:55.970508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:55.988037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:55.989245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:56.047228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:56.054362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 578/1148] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:56.077267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a641fb-d5b6-4e8a-b496-4bc1c6850d7e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:56.176604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a641fb-d5b6-4e8a-b496-4bc1c6850d7e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:56.199168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:56.209838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:56.210469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:56.210751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:56.211022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:56.221834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:56.226393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:56.227478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:08:56.228402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 571/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:08:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:56.234202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:56.328311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9932a4-a17c-4385-92da-021db756d906 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:56.360438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:56.361056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:56.361337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:56.361606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:56.362259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:56.362560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:56.363916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:56.371546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 579/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:56.377614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:56.475849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aaff7-3d08-4e63-b133-f8fcb3419118 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:56.509294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:56.509914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:56.510194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:56.510466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] PATH_INFO: `/v3/users/bfe16a7f8f97488e812b13319e98c3d6` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:56.511566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] RBAC: Authorizing `identity:get_user(user_id=bfe16a7f8f97488e812b13319e98c3d6)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:56.513063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:56.514629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 572/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:56 2022] GET /identity/v3/users/bfe16a7f8f97488e812b13319e98c3d6 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:56.520335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:56.578821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:56.619276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4997e-009a-4bd1-9683-59a07259b444 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:56.656541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b781d7-3f58-415a-ba09-cc2456fb021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:56.657212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b781d7-3f58-415a-ba09-cc2456fb021d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:56.657492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b781d7-3f58-415a-ba09-cc2456fb021d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:56.657762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b781d7-3f58-415a-ba09-cc2456fb021d None admin] PATH_INFO: `/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:56.659366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b781d7-3f58-415a-ba09-cc2456fb021d None admin] RBAC: Authorizing `identity:get_project(project_id=1f6a02dfdd7840dda1ed7e7934c3880d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:56.660814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b781d7-3f58-415a-ba09-cc2456fb021d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:56.662587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 580/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:56 2022] GET /identity/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:56.669044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:56.735088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:56.791090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5a676-5978-45d8-8189-0dc6cf7dee16 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:56.838710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919faa53-d23f-4d94-b746-8d082d348a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:56.839368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919faa53-d23f-4d94-b746-8d082d348a82 None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:56.839651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919faa53-d23f-4d94-b746-8d082d348a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:56.839927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919faa53-d23f-4d94-b746-8d082d348a82 None admin] PATH_INFO: `/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d/users/bfe16a7f8f97488e812b13319e98c3d6/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:56.842971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919faa53-d23f-4d94-b746-8d082d348a82 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f6a02dfdd7840dda1ed7e7934c3880d, user_id=bfe16a7f8f97488e812b13319e98c3d6, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:56.844398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919faa53-d23f-4d94-b746-8d082d348a82 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:56.924578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 573/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:08:56 2022] PUT /identity/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d/users/bfe16a7f8f97488e812b13319e98c3d6/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:08:58.994499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbd8d2-2f14-4959-9bef-1d6b595b6e77 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:58.995000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbd8d2-2f14-4959-9bef-1d6b595b6e77 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:58.995000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbd8d2-2f14-4959-9bef-1d6b595b6e77 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:58.996308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 581/1154] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:08:59.001836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:59.002298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:59.002391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:59.021505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:08:59.022685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:08:59.086930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 574/1155] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:08:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:08:59.110284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbd8d2-2f14-4959-9bef-1d6b595b6e77 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:59.210900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbd8d2-2f14-4959-9bef-1d6b595b6e77 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:59.235835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:59.243857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:59.244467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:59.244790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:59.245079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:59.256853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:08:59.261430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:08:59.263935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:08:59.264779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 582/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:08:59.269514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:59.311357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:59.365728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e2967-e667-4efe-9ed9-75f38ba386b2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:59.399291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:59.399891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:59.400169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:59.400437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:59.401120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:08:59.401420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:59.402768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:59.411476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 575/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:59.417227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:59.511979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e148af3-b10d-4f59-a2b6-48ed150d4245 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:59.544751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:59.545367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:59.545642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:59.545908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] PATH_INFO: `/v3/users/bfe16a7f8f97488e812b13319e98c3d6` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:59.547001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] RBAC: Authorizing `identity:get_user(user_id=bfe16a7f8f97488e812b13319e98c3d6)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:59.548403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:59.549979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 583/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:08:59 2022] GET /identity/v3/users/bfe16a7f8f97488e812b13319e98c3d6 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:59.555792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:59.655489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4f1d-4914-4067-8a3b-6ab7726dbe5c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:59.657829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:59.688707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:59.689356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:59.689796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:59.690344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:59.755117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:59.756538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:59.850028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 576/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:08:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:08:59.856041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:08:59.911599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:59.954751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c029-f71f-4129-b8bd-3482a1181b76 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:08:59.987065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e87e-303e-4092-a409-d175eae8e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:08:59.987676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e87e-303e-4092-a409-d175eae8e899 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:08:59.987951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e87e-303e-4092-a409-d175eae8e899 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:08:59.988218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e87e-303e-4092-a409-d175eae8e899 None admin] PATH_INFO: `/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:08:59.989338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95e87e-303e-4092-a409-d175eae8e899 None admin] RBAC: Authorizing `identity:get_project(project_id=1f6a02dfdd7840dda1ed7e7934c3880d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:08:59.990738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95e87e-303e-4092-a409-d175eae8e899 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:08:59.992330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 584/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:08:59 2022] GET /identity/v3/projects/1f6a02dfdd7840dda1ed7e7934c3880d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:08:59.998808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:00.001815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:00.094610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f044aca-20c3-4af7-8eea-547e389c779c None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:00.127803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:00.128339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:00.128618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:00.128889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:00.129638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=1f6a02dfdd7840dda1ed7e7934c3880d, user.id=bfe16a7f8f97488e812b13319e98c3d6) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:00.129945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:00.131347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea17c4-3caa-4c94-bfc0-27011c45a972 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:00.136889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 577/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:08:59 2022] GET /identity/v3/role_assignments?user.id=bfe16a7f8f97488e812b13319e98c3d6&scope.project.id=1f6a02dfdd7840dda1ed7e7934c3880d&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:02.119933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c376-eff8-4846-98e2-eb1363e97dae None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:02.120453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c376-eff8-4846-98e2-eb1363e97dae None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:02.120453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c376-eff8-4846-98e2-eb1363e97dae None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:02.121834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 585/1162] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:09:02.127524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:02.127958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:02.128078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:02.146376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:02.147589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:02.208300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:02.213570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 578/1163] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:02.235455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c376-eff8-4846-98e2-eb1363e97dae None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:02.334341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c376-eff8-4846-98e2-eb1363e97dae None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:02.355328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:02.366757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:02.367352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:02.367630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:02.367901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:02.432155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:02.433547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:02.527329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 586/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:09:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:09:02.533804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:02.633403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4d52-97a0-4957-8b8d-801a668b8087 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:02.668005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:02.668637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:02.669651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:02.670057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:02.676481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:02.677951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-52fac7a8-3b02-475c-8d2d-8f93386e9bb2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:09:02.689054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:09:02.690311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:09:02.691337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 579/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:09:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:09:02.696060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:02.719366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:02.792702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71012206-cae8-4591-b1db-9df640a2a8f1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:02.825822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:02.826420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:02.826699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:02.826971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:02.827604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:02.827906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:02.829280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99f5b-3011-4ec1-8848-5973febd0c13 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:02.836120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 587/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 10 13:09:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:04.814618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d235ed-a676-45e7-be33-02b4b2016d50 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:04.815149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d235ed-a676-45e7-be33-02b4b2016d50 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:04.815149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d235ed-a676-45e7-be33-02b4b2016d50 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:04.816504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 580/1167] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:09:04.821518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301ad0b-a4f2-4968-8562-08d287a68ecc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:04.821934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301ad0b-a4f2-4968-8562-08d287a68ecc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:04.822068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301ad0b-a4f2-4968-8562-08d287a68ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:04.840316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e301ad0b-a4f2-4968-8562-08d287a68ecc None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:04.841546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e301ad0b-a4f2-4968-8562-08d287a68ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:04.906011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 588/1168] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:04.928637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d235ed-a676-45e7-be33-02b4b2016d50 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:04.987962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:05.027970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d235ed-a676-45e7-be33-02b4b2016d50 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:05.061841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803ebd4-f0b3-4869-91b9-dbcdd735a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:05.062451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ebd4-f0b3-4869-91b9-dbcdd735a48e None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:05.062732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ebd4-f0b3-4869-91b9-dbcdd735a48e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:05.063005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ebd4-f0b3-4869-91b9-dbcdd735a48e None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:05.063686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803ebd4-f0b3-4869-91b9-dbcdd735a48e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:05.065066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803ebd4-f0b3-4869-91b9-dbcdd735a48e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:05.128336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 581/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Wed Aug 10 13:09:04 2022] POST /identity/v3/domains => generated 267 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:09:07.148132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02f561-064e-45f3-8857-6395fadbb6c8 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:07.148682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02f561-064e-45f3-8857-6395fadbb6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:07.148682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02f561-064e-45f3-8857-6395fadbb6c8 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:07.150070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 589/1170] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:09:07.157160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:07.157565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:07.157709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:07.175495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:07.176661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:07.241142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 582/1171] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:07.265678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02f561-064e-45f3-8857-6395fadbb6c8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:07.283479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:07.365191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02f561-064e-45f3-8857-6395fadbb6c8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:07.398382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a4e1-e50b-46e1-96a7-be7b35bf9431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:07.398941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a4e1-e50b-46e1-96a7-be7b35bf9431 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:07.399215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a4e1-e50b-46e1-96a7-be7b35bf9431 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:07.399483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a4e1-e50b-46e1-96a7-be7b35bf9431 None admin] PATH_INFO: `/v3/domains/3bafc561ff634f71afa408ceefd39627` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:07.410954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26a4e1-e50b-46e1-96a7-be7b35bf9431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3bafc561ff634f71afa408ceefd39627)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:07.412371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26a4e1-e50b-46e1-96a7-be7b35bf9431 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:07.414072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 590/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Aug 10 13:09:07 2022] GET /identity/v3/domains/3bafc561ff634f71afa408ceefd39627 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:07.420332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:07.469552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:07.518540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992fee7-ad57-4763-98ab-d99c3de71b84 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:07.552138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406205a-d8af-4f86-9f9c-8db086d88199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:07.552741 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406205a-d8af-4f86-9f9c-8db086d88199 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:07.553047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406205a-d8af-4f86-9f9c-8db086d88199 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:07.553322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406205a-d8af-4f86-9f9c-8db086d88199 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:07.553965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406205a-d8af-4f86-9f9c-8db086d88199 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:07.555389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406205a-d8af-4f86-9f9c-8db086d88199 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:07.648079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 583/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:09:07 2022] POST /identity/v3/projects => generated 361 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:09:09.686985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0dae2-7cd5-4892-af8a-1f011cffc3d5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:09.687455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0dae2-7cd5-4892-af8a-1f011cffc3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:09.687548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0dae2-7cd5-4892-af8a-1f011cffc3d5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:09.688875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 591/1174] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:09:09.693984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:09.694391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:09.694542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:09.712946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:09.714167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:09.778757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 584/1175] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:09.803841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0dae2-7cd5-4892-af8a-1f011cffc3d5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:09.879645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:09.902671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0dae2-7cd5-4892-af8a-1f011cffc3d5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:09.935305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98996c-02a4-4789-ac01-3e40bd7dbe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:09.935852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98996c-02a4-4789-ac01-3e40bd7dbe76 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:09.936132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98996c-02a4-4789-ac01-3e40bd7dbe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:09.936406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98996c-02a4-4789-ac01-3e40bd7dbe76 None admin] PATH_INFO: `/v3/domains/3bafc561ff634f71afa408ceefd39627` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:09.937530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98996c-02a4-4789-ac01-3e40bd7dbe76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3bafc561ff634f71afa408ceefd39627)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:09.938949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98996c-02a4-4789-ac01-3e40bd7dbe76 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:09.940511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 592/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Aug 10 13:09:09 2022] GET /identity/v3/domains/3bafc561ff634f71afa408ceefd39627 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:09.946286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:09.976230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:10.043883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9ff4-e125-4832-9117-b9ac26779cdd None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:10.076199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7bb7d-2075-41e8-8e46-c1f33b8fa799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:10.076787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7bb7d-2075-41e8-8e46-c1f33b8fa799 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:10.077098 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7bb7d-2075-41e8-8e46-c1f33b8fa799 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:10.077372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7bb7d-2075-41e8-8e46-c1f33b8fa799 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:10.078003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7bb7d-2075-41e8-8e46-c1f33b8fa799 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:10.079421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7bb7d-2075-41e8-8e46-c1f33b8fa799 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:10.178402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 585/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:09:09 2022] POST /identity/v3/users => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:09:12.208838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf709-ea1e-45d3-9c74-89bbc55a6506 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:12.209458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf709-ea1e-45d3-9c74-89bbc55a6506 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:12.209458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf709-ea1e-45d3-9c74-89bbc55a6506 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:12.210755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 593/1178] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:09:12.216579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:12.217039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:12.217152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:12.234507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:12.235680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:12.299369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 586/1179] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:12.322449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdf709-ea1e-45d3-9c74-89bbc55a6506 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:12.421414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdf709-ea1e-45d3-9c74-89bbc55a6506 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:12.453351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:12.453967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:12.454246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:12.454524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:12.464561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:09:12.469947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:09:12.471108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:09:12.472176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 594/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:09:12.477062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:12.494701 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:12.573923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609742f-4d8c-4a3d-a1c0-771e4e60932b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:12.607174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:12.608153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:12.608614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:12.609088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:12.610161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:12.610645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:12.612811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:12.624674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 587/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:09:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:12.630427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:12.634140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:12.730901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65992b-d64f-4386-be12-39f48d6770d0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:12.763926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:12.764544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:12.764823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:12.765126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] PATH_INFO: `/v3/users/4efa8e1192fc4243aa009f171c175fda` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:12.777366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] RBAC: Authorizing `identity:get_user(user_id=4efa8e1192fc4243aa009f171c175fda)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:12.779915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:12.781519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 595/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:09:12 2022] GET /identity/v3/users/4efa8e1192fc4243aa009f171c175fda => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:12.787287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:12.864148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:12.884864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd189-bf63-430f-bbc3-1a690445ca1f None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:12.918577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:12.919745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:12.920021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:12.920287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:12.985431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:12.986784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:13.058719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:13.079859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 588/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:09:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:09:13.085838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:13.146179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:13.184657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8967-85d4-40d4-a4b2-bf78d80206e4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:13.217176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0c68e-d2cb-46b1-b23d-9d19915a08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:13.217793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c68e-d2cb-46b1-b23d-9d19915a08a2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:13.218067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c68e-d2cb-46b1-b23d-9d19915a08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:13.218338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c68e-d2cb-46b1-b23d-9d19915a08a2 None admin] PATH_INFO: `/v3/projects/89e0cca139f549bca2ca0a5d95e361e8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:13.219436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0c68e-d2cb-46b1-b23d-9d19915a08a2 None admin] RBAC: Authorizing `identity:get_project(project_id=89e0cca139f549bca2ca0a5d95e361e8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:13.220847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0c68e-d2cb-46b1-b23d-9d19915a08a2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:13.222480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 596/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:09:13 2022] GET /identity/v3/projects/89e0cca139f549bca2ca0a5d95e361e8 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:13.228536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:13.322280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e9a61-fc63-41ea-afd1-a202d243a09e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:13.355223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:13.355808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:13.356161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:13.356435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:13.357174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=89e0cca139f549bca2ca0a5d95e361e8, user.id=4efa8e1192fc4243aa009f171c175fda) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:13.357482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:13.358876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccdb0e-7e22-403c-bded-7380ecc25f65 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:13.363966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 589/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:09:13 2022] GET /identity/v3/role_assignments?user.id=4efa8e1192fc4243aa009f171c175fda&scope.project.id=89e0cca139f549bca2ca0a5d95e361e8&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:15.320049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5b25-ac6c-4b07-b86c-ae496c1a9fca None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:15.320596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5b25-ac6c-4b07-b86c-ae496c1a9fca None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:15.320596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5b25-ac6c-4b07-b86c-ae496c1a9fca None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:15.321968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 597/1186] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:09:15.327510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:15.327939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:15.328049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:15.346135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:15.347311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:15.380553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:15.410912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 590/1187] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:15.432524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5b25-ac6c-4b07-b86c-ae496c1a9fca None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:15.536707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5b25-ac6c-4b07-b86c-ae496c1a9fca None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:15.564733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:15.570299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:15.570873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:15.571153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:15.571426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:15.582228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:09:15.586904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:09:15.588001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:09:15.589062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 598/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:09:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:09:15.594174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:15.692437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f76c-3f59-4656-85e9-1c511975c699 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:15.724863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:15.725492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:15.725774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:15.726049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:15.726710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:15.727013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:15.728433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:15.736292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 591/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:09:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:15.742138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:15.838607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3662305-d7c9-4d54-9ac6-dec157bb2829 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:15.871010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:15.871649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:15.871934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:15.872206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] PATH_INFO: `/v3/users/4efa8e1192fc4243aa009f171c175fda` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:15.873342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] RBAC: Authorizing `identity:get_user(user_id=4efa8e1192fc4243aa009f171c175fda)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:15.874752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:15.876307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 599/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:09:15 2022] GET /identity/v3/users/4efa8e1192fc4243aa009f171c175fda => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:15.881943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:15.916973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:15.979853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5771716-8c77-4ab1-9b8d-635dfa8e6807 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:16.013146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63285831-11a4-40e9-94ce-eb5eda67f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:16.013755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63285831-11a4-40e9-94ce-eb5eda67f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:16.014030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63285831-11a4-40e9-94ce-eb5eda67f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:16.014298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63285831-11a4-40e9-94ce-eb5eda67f6a2 None admin] PATH_INFO: `/v3/projects/89e0cca139f549bca2ca0a5d95e361e8` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:16.015384 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63285831-11a4-40e9-94ce-eb5eda67f6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=89e0cca139f549bca2ca0a5d95e361e8)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:16.016892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63285831-11a4-40e9-94ce-eb5eda67f6a2 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:16.018519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 592/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:09:15 2022] GET /identity/v3/projects/89e0cca139f549bca2ca0a5d95e361e8 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:16.024361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:16.081013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:16.120553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f8db4-d8cc-4408-b26a-bf37b179d041 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:16.154661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e7a06-c8f3-4d63-8967-d9f3907ad683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:16.155311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7a06-c8f3-4d63-8967-d9f3907ad683 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:16.155592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7a06-c8f3-4d63-8967-d9f3907ad683 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:16.155867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7a06-c8f3-4d63-8967-d9f3907ad683 None admin] PATH_INFO: `/v3/projects/89e0cca139f549bca2ca0a5d95e361e8/users/4efa8e1192fc4243aa009f171c175fda/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:16.157805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e7a06-c8f3-4d63-8967-d9f3907ad683 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e0cca139f549bca2ca0a5d95e361e8, user_id=4efa8e1192fc4243aa009f171c175fda, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:16.159217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e7a06-c8f3-4d63-8967-d9f3907ad683 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:16.232371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 600/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:09:16 2022] PUT /identity/v3/projects/89e0cca139f549bca2ca0a5d95e361e8/users/4efa8e1192fc4243aa009f171c175fda/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:09:18.185245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6645a8-f8f1-4804-b6f2-5076e77bb3b3 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:18.185808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6645a8-f8f1-4804-b6f2-5076e77bb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:18.185808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6645a8-f8f1-4804-b6f2-5076e77bb3b3 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:18.187120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 593/1193] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:09:18.192246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:18.192626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:18.192800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:18.210948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:18.212117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:18.275657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 601/1194] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:09:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:09:18.299515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6645a8-f8f1-4804-b6f2-5076e77bb3b3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:18.398759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6645a8-f8f1-4804-b6f2-5076e77bb3b3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:18.400239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:18.431967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:18.432585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:18.432864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:18.433168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:18.443696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:09:18.448612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:09:18.453301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:09:18.454314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 594/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:09:18.456607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:18.503158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:18.552591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb491a-8eb5-48a3-b641-4347921f7f4b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:18.585145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:18.585732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:18.586007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:18.586274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:18.586922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:18.587220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:18.588565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:18.596899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 602/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:18.602563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:18.705825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e5b4-6abb-49d0-8a9d-201722c61481 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:18.739853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:18.740477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:18.740759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:18.741067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] PATH_INFO: `/v3/users/4efa8e1192fc4243aa009f171c175fda` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:18.742176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] RBAC: Authorizing `identity:get_user(user_id=4efa8e1192fc4243aa009f171c175fda)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:18.743598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:18.745184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 595/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 10 13:09:18 2022] GET /identity/v3/users/4efa8e1192fc4243aa009f171c175fda => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:18.751623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:18.848311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462ee0-cd48-4145-b32e-4e9ef9dffabf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:18.858574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:18.880784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:18.881412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:18.881697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:18.882021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:18.946520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:18.947941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:19.044678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 603/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:09:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:09:19.050534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:19.081142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:19.150938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41988-c1fa-426c-adbc-4b0e2c0deb82 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:19.184805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7643a-e28c-4b1a-9dde-03eb96a0e2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:19.185444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7643a-e28c-4b1a-9dde-03eb96a0e2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:19.185972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7643a-e28c-4b1a-9dde-03eb96a0e2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:19.185972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7643a-e28c-4b1a-9dde-03eb96a0e2a5 None admin] PATH_INFO: `/v3/projects/89e0cca139f549bca2ca0a5d95e361e8` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:19.187073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7643a-e28c-4b1a-9dde-03eb96a0e2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=89e0cca139f549bca2ca0a5d95e361e8)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:19.188482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7643a-e28c-4b1a-9dde-03eb96a0e2a5 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:19.190122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 596/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 10 13:09:19 2022] GET /identity/v3/projects/89e0cca139f549bca2ca0a5d95e361e8 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:19.196185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:19.208009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:19.290869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c16b5-0071-4a02-8788-0d2fbdbac689 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:19.322785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:19.323313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:19.323587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:19.323854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:19.324560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=89e0cca139f549bca2ca0a5d95e361e8, user.id=4efa8e1192fc4243aa009f171c175fda) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:09:19.324921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:19.326347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeca04-4d8e-4fd6-845c-ef3d946eccaa None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:19.331817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 604/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:09:19 2022] GET /identity/v3/role_assignments?user.id=4efa8e1192fc4243aa009f171c175fda&scope.project.id=89e0cca139f549bca2ca0a5d95e361e8&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:09:29.180258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=51 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:29.329290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=63 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:31.951737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5d93-db38-4368-a24a-7a38bc3a63ab None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:31.952284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5d93-db38-4368-a24a-7a38bc3a63ab None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:31.952284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5d93-db38-4368-a24a-7a38bc3a63ab None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:31.953562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 597/1201] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:31.958678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b07567-739e-40a5-acce-d022b8941131 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:31.961436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b07567-739e-40a5-acce-d022b8941131 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:31.961436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b07567-739e-40a5-acce-d022b8941131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:31.976787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-93b07567-739e-40a5-acce-d022b8941131 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:31.977999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-93b07567-739e-40a5-acce-d022b8941131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:32.042315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 605/1202] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:32.067065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5d93-db38-4368-a24a-7a38bc3a63ab None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:32.168607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5d93-db38-4368-a24a-7a38bc3a63ab None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:32.201488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:32.202198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:32.202427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:32.202697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:32.212808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:32.214557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:32.228141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:32.229347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:32.230470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:32.230470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:32.230470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:32.230470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 598/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:32.235708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b07567-739e-40a5-acce-d022b8941131 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:32.336722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b07567-739e-40a5-acce-d022b8941131 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:32.365696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:32.371765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:32.375820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:32.375820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:32.375820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:32.375820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:32.375820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:32.375820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42c72-9c85-4919-b2e5-8dc441593432 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:32.387060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 606/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:32.392829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:32.450972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:32.500678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831825-0557-4086-a294-55a5ac1c3604 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:32.536279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f14a7-66e8-47af-8ee2-2abcfc7855e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:32.537118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f14a7-66e8-47af-8ee2-2abcfc7855e0 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:32.537396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f14a7-66e8-47af-8ee2-2abcfc7855e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:32.537664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f14a7-66e8-47af-8ee2-2abcfc7855e0 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:32.538321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f14a7-66e8-47af-8ee2-2abcfc7855e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:32.539941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f14a7-66e8-47af-8ee2-2abcfc7855e0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:32.678174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 599/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 10 13:10:32 2022] POST /identity/v3/users => generated 251 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:10:34.732546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132331-1d1e-4a21-a06e-e68df92d5602 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:34.733120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132331-1d1e-4a21-a06e-e68df92d5602 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:34.733223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132331-1d1e-4a21-a06e-e68df92d5602 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:34.734528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 607/1206] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:34.739627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:34.740058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:34.740166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:34.758609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:34.759792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:34.825147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 600/1207] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:34.848048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24132331-1d1e-4a21-a06e-e68df92d5602 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:34.949342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24132331-1d1e-4a21-a06e-e68df92d5602 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:34.981626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:34.982250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:34.982531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:34.982804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:34.993153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:10:34.998643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:34.999737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:35.000884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 608/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:10:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:35.005863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:35.044633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:35.102860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6bd3-7c62-492e-a69c-1df6ea452379 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:35.137330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:35.137934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:35.138211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:35.138481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:35.139136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:35.139436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:35.140784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:35.148322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 601/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:10:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:35.154194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:35.203460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:35.253119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d7d38-600e-44a8-81b4-60c09b43bb72 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:35.286643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:35.287304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:35.287585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:35.287858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:35.299104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:35.300534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:35.314274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:35.315442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:35.316550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:35.316550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:35.316550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:35.316550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 609/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:35.321203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:35.417389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:35.420252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd549e3-94e4-4719-a6a7-fe64dd6d1996 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:35.453366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:35.454021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:35.454258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:35.454531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:35.455174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:35.455476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:35.456880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:35.469636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 602/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:35.475295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:35.574700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f035af-4dcb-490b-b277-408ae5577e1a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:35.576195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:35.607435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:35.608091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:35.608372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:35.608647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:35.615850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:35.617305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:10:35.628642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:35.629838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:35.631026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 610/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:35.635587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:35.739088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3db0d-dae0-4272-bc95-73cc21b41bf7 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:35.771566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:35.772188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:35.772430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:35.772698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:35.773648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:35.774181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:35.777115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:35.794237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 603/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 10 13:10:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:35.800039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:35.907604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b77365-adb4-4813-aef2-907772fe6ae7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:35.940172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:35.940768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:35.941072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:35.941343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:35.970272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:36.009626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:36.011858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:36.111238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 611/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:10:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:10:36.117050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:36.136297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:36.216529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cfbdf-ebcd-48a5-98e9-c9503fa279d1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:36.251095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:36.251742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:36.252035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:36.252297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:36.262493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:36.263915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:36.278294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:36.279756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:36.280650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:36.280650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:36.280650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:36.280650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 604/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:36.285437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:36.347902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:36.389400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7703873-3513-4634-b2a1-343c47b97bed None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:36.421792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:36.422395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:36.422673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:36.422938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:36.423570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:36.423872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:36.425244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:36.437073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 612/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:36.442854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:36.511538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:36.541532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9b0a-4fad-49d1-8070-3a882b565ff4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:36.580201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:36.580823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:36.581130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:36.581401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:36.587705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:36.589801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:10:36.600602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:36.605928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:36.607035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 605/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:36.608549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:36.709290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c22f9-deab-46bf-ae69-ea1c809100f3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:36.731803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:36.742533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:36.743137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:36.743444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:36.743687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:36.744368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:36.744673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:36.746641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6621c-0a90-410d-b3a1-5c60514daee9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:36.757766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 613/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:10:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:36.764289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:36.864193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43778e-0a02-4f47-8654-8376868dae98 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:36.891740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:36.897343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:36.897945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:36.898224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:36.898493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:36.899215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=8c82e23142a343b096a493e01071a506) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:36.899516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:36.900943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d63d30-358f-42bd-a667-06e96560b242 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:36.906487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 606/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:10:36 2022] GET /identity/v3/role_assignments?user.id=8c82e23142a343b096a493e01071a506&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:38.882582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e7ec-1e11-42c3-bbc6-645a570596a9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:38.883097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e7ec-1e11-42c3-bbc6-645a570596a9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:38.883186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e7ec-1e11-42c3-bbc6-645a570596a9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:38.884519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 614/1220] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:38.890419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:38.890808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:38.890967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:38.909320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:38.910386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:38.975425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 607/1221] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:38.997993 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44e7ec-1e11-42c3-bbc6-645a570596a9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:39.098548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44e7ec-1e11-42c3-bbc6-645a570596a9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:39.130291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:39.130916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:39.131197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:39.131471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:39.142054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:10:39.147415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:39.148480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:39.149515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 615/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:10:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:39.154921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:39.251654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08a77-e459-405b-aeef-4c7edae96235 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:39.274511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:39.285331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:39.285966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:39.286252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:39.286530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:39.287194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] RBAC: Adding query filter params (name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:39.287506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:39.288867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:39.297169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 608/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:39.302986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:39.346440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:39.401732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8662d-f0e6-4be5-9d0b-50f424fd4214 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:39.435079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:39.435752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:39.436035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:39.436307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:39.447622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:39.453723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:39.463505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:39.464627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:39.465786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:39.465786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:39.465786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:39.465786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 616/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:39.470597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:39.569910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152dd2f-3b5a-4567-b5a6-374e96f4acff None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:39.603188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:39.603778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:39.604054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:39.604324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:39.604956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:39.605695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:39.607050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:39.618439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 609/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:39.624844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:39.718076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:39.724310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee98f2-043a-465a-8c54-4d950976257f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:39.756662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:39.757315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:39.757591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:39.757860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:39.765998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:39.767388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:10:39.778375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:39.779756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:39.780729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:39.784415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 617/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:39.785558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:39.833315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:39.886406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f96b2-e81a-420e-952c-cd22047822e3 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:39.920067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:39.920663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:39.920927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:39.921222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:39.921938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:39.922238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:39.923643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:39.941501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 610/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 10 13:10:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:10:39.947133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:40.046363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2fd9d-89cd-414e-a38b-63d0cda453bb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:40.079108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:40.079727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:40.080004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:40.080275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:40.090522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:40.091895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:40.106213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:40.107400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:40.108540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:40.108540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:40.108540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:40.108540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 618/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:40.113053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:40.210820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859fa98-aed2-43d4-9597-e10100534490 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:40.213203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:40.245228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:40.246009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:40.246293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:40.246565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:40.247195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:40.247500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:40.248849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:40.259587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 611/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:40.265483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:40.274647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:40.363282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdb552-db33-4840-b449-6be16c5a1183 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:40.394981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:40.395607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:40.395887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:40.396157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:40.403231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:40.404904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:10:40.415677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:40.416746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:40.422643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 619/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:40.423457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:40.522667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4bbce-1c76-4cc7-918c-47e342364904 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:40.555302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:40.556016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:40.556296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:40.556568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:40.557284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:40.557588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:40.559603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77256710-1301-4343-b67e-ef7b7c2bce29 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:40.571292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 612/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:40.577539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:40.642263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:40.677914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d441bb-33f9-49d3-83c3-45461b783fc3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:40.711331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcd05e-1b2e-4576-8a19-b80cc31d3d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:40.711993 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcd05e-1b2e-4576-8a19-b80cc31d3d12 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:40.712274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcd05e-1b2e-4576-8a19-b80cc31d3d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:40.712555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcd05e-1b2e-4576-8a19-b80cc31d3d12 None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/8c82e23142a343b096a493e01071a506/roles/20ed2205b8374fa981d59dd07298760c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:40.727961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcd05e-1b2e-4576-8a19-b80cc31d3d12 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=8c82e23142a343b096a493e01071a506, role_id=20ed2205b8374fa981d59dd07298760c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:40.729452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcd05e-1b2e-4576-8a19-b80cc31d3d12 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:40.802971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 620/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:10:40 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/8c82e23142a343b096a493e01071a506/roles/20ed2205b8374fa981d59dd07298760c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:10:42.780312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01910941-3dde-4cfa-839f-0a3e2d83457c None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:42.780849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01910941-3dde-4cfa-839f-0a3e2d83457c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:42.780940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01910941-3dde-4cfa-839f-0a3e2d83457c None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:42.782277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 613/1233] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:42.787322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:42.787725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:42.787867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:42.805991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:42.807197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:42.872133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 621/1234] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:42.895107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01910941-3dde-4cfa-839f-0a3e2d83457c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:42.920909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:42.996185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01910941-3dde-4cfa-839f-0a3e2d83457c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:43.032638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:43.033309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:43.033639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:43.033917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:43.049720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:10:43.054602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:43.055705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:43.056663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 614/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 10 13:10:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:43.062629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:43.107370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:43.159367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d559d-e65c-4d1e-b350-2a1f3db75bf7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:43.192319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:43.192853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:43.193167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:43.193445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:43.194106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:43.194408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:43.195760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:43.203821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 622/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 10 13:10:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:43.209785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:43.299293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:43.314276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08cf1-da71-4031-9ca7-0e0244090fb4 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:43.354676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:43.356476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:43.356848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:43.357244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:43.367942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:43.369527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:43.385243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:43.391431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:43.392628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:43.392628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:43.392628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:43.392628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 615/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:43.392926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:43.490598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b03b1-24be-408a-894c-05103b259ed6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:43.499668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:43.522889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:43.523494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:43.523771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:43.524042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:43.524673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:43.525011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:43.526391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:43.537982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 623/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:43.544292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:43.645531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26292d70-b77b-4fe0-97f3-b56393525f1e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:43.681941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:43.682661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:43.682940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:43.684499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:43.687197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:43.692165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:43.693731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:10:43.705033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:43.706191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:43.707413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 616/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:10:43.712234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:43.810561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333865-df4d-4033-ada1-e64b16b92d2d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:43.844060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:43.844664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:43.844944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:43.845245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:43.845965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:43.846273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:43.847681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:43.865181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 624/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 10 13:10:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:43.872278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:43.972102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed17f6e-4fbf-4aff-a907-cc0a0ea07624 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:44.006087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:44.006685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:44.006962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:44.007231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:44.066814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:44.073082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:44.075106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:44.168164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 617/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:10:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:10:44.174118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:44.201478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:44.270714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbd68a-0b6c-48c2-b4f2-a222798ed699 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:44.304360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:44.305019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:44.305330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:44.305604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:44.315987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:44.317627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:44.332543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:44.333690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:44.334811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:44.334811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:44.334811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:44.334811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 625/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:44.340441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:44.431824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:44.437940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd9843-e84d-462f-9cc9-6ad25f7bc16e None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:44.472186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:44.472789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:44.473095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:44.473473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:44.474113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:44.474411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:44.475756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:44.486871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 618/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:44.492562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:44.574993 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:44.595836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3f821-3190-4016-ae14-325f67bf46a2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:44.627643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:44.628269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:44.628548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:44.628814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:44.635135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:44.637193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:10:44.648104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:44.649171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:44.650248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 626/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:44.655959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:44.753144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc91f-163d-4881-932a-2c7d223d03c0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:44.786368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:44.786971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:44.787251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:44.787524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:44.788208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:44.788515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:44.790029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e379102-9c75-47bd-82b6-fcf46a92b632 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:44.801945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 619/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:44.808547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:44.905829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4c318-4f85-450d-bda6-5a4153d7c1d3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:44.939152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:44.939744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:44.940023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:44.940290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:44.941026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] RBAC: Adding query filter params (role.id=20ed2205b8374fa981d59dd07298760c, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=8c82e23142a343b096a493e01071a506) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:44.941327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:44.942718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54c26a-c447-4c46-8fbf-7ebc826da4d1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:44.948325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 627/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:10:44 2022] GET /identity/v3/role_assignments?user.id=8c82e23142a343b096a493e01071a506&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=20ed2205b8374fa981d59dd07298760c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:46.993651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103945-567f-41ba-b225-4e5f022251ca None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:46.994162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103945-567f-41ba-b225-4e5f022251ca None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:46.994162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103945-567f-41ba-b225-4e5f022251ca None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:46.995499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 620/1247] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:47.000769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:47.002778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:47.002778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:47.007566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:47.019379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:47.020563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:47.085722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 628/1248] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:47.109961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68103945-567f-41ba-b225-4e5f022251ca None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:47.129064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:47.211985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68103945-567f-41ba-b225-4e5f022251ca None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:47.245278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:47.246381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:47.246666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:47.246940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:47.256862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:10:47.262364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:47.263481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:47.264648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 621/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:10:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:47.269535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:47.366570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a31794-7768-499a-a41b-6478ea256ee2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:47.394934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:47.399682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:47.400280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:47.400558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:47.400829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:47.401544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:47.401912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:47.403246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:47.411223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 629/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:47.417137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:47.494960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:47.518064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea134643-dc34-4b12-b62b-c8dfc8fab143 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:47.550478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:47.551093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:47.551369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:47.551638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:47.562204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:47.563621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:47.577572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:47.584486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:47.585641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:47.585641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:47.585641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:47.585641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 622/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:47.585937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:47.682199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158d17-7923-411a-8b12-07d24647f310 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:47.713771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:47.715000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:47.715306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:47.715637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:47.716268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:47.716567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:47.717948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:47.729130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 630/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:47.735986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:47.836145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab8247-d46e-4500-911e-a310138cab43 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:47.862894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:47.871536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:47.872168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:47.872450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:47.872722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:47.885332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:47.887213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:10:47.898426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:47.899581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:47.900694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 623/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:47.907102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:47.960227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:48.002197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf6798-ccd9-481e-8214-05ff7c189c58 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:48.035144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:48.035740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:48.036020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:48.036300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:48.037247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:48.037710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:48.039733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:48.056866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 631/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 10 13:10:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:48.062978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:48.161549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e8b70-c173-4e16-80e6-0cb09ac9ebcf None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:48.193999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:48.194605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:48.194886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:48.195161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:48.250469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:48.262521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:48.263967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:48.364500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 624/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:10:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:10:48.371036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:48.475657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0251989-56b8-409d-a832-20fee53a8660 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:48.489272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:48.509707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:48.510334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:48.510611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:48.510882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:48.521154 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:48.522593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:48.537091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:48.538067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:48.538963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:48.538963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:48.538963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:48.538963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 632/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:48.544259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:48.630153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:48.642706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb11e9-a014-49b1-b430-9aa5afa14034 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:48.679358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:48.679964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:48.680245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:48.680518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:48.681180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:48.681483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:48.683920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:48.695263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 625/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:48.702120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:48.801258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb27e-7afd-426c-8f1f-b45c43576516 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:48.833534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:48.834154 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:48.834431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:48.834702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:48.841136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:48.842571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:10:48.853997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:48.856225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:48.857457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 633/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:10:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:48.861644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:48.959614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d9eb8-0fc9-4afc-b18f-c83608917465 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:48.993152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:48.993754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:48.994032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:48.994302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:48.995099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:48.995407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:48.997404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545639b0-cdb8-4e3f-bed1-fdb33a097811 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:49.008804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 626/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:49.015565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:49.041839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:49.114094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0eb3e-ef30-4987-9894-da58ad01ac39 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:49.147705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:49.148309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:49.148613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:49.148886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:49.149636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=8c82e23142a343b096a493e01071a506) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:49.149943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:49.151361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee371ba-11a8-4fe7-96dc-8d07650f8799 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:49.157706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 634/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:10:49 2022] GET /identity/v3/role_assignments?user.id=8c82e23142a343b096a493e01071a506&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=dd76871a868143b482c4982027fdc1f5 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:10:51.130081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fd126-59bf-4b7c-ba34-870261445d64 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:51.130636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fd126-59bf-4b7c-ba34-870261445d64 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:51.130636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fd126-59bf-4b7c-ba34-870261445d64 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:51.131969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 627/1261] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:51.137437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:51.137862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:51.137992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:51.156504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:51.157788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:51.224889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 635/1262] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:51.247867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fd126-59bf-4b7c-ba34-870261445d64 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:51.259000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:51.348385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fd126-59bf-4b7c-ba34-870261445d64 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:51.383682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:51.384244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:51.384527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:51.384801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:51.395324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:10:51.399999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:51.401311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:51.403045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 628/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:10:51.408178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:51.426323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:51.518520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2093b2c-d692-4775-93d6-c42fbd19fa30 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:51.551752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:51.552357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:51.552649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:51.552900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:51.553595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:51.553897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:51.555255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:51.563785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 636/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:51.569675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:51.635177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:51.667753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67987def-050f-48be-8e91-8945d8c55804 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:51.701935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:51.702559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:51.702838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:51.703107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:51.713889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:51.715534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:51.729640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:51.731084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:51.732246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:51.732246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:51.732246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:51.732246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 629/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:51.736525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:51.821353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:51.843410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d6b50-6b81-4a02-be4e-740fb37f6e28 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:51.875895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:51.876488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:51.876764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:51.877063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:51.877688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:51.877984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:51.879350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:51.892952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 637/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:10:51.897300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.001957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096cfbc-0c02-443a-b6b1-6fb3007ec3c0 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:52.020710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:52.034367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:52.034989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:52.035266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:52.035532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:52.043959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:52.045440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:10:52.055931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:52.062175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:52.063364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 630/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:52.064086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.169124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427acbd-b435-4ac1-a660-26663f6419ae None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:52.202220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:52.202859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:52.203197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:52.203469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:52.204183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:52.204485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:52.205917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:52.215502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:52.223482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 638/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 10 13:10:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:52.229272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.333592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da13ae3-46c0-4ab2-baa1-e9446c3c5cb9 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:52.367393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:52.368081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:52.368402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:52.368721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:52.379312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:52.380737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:52.394297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:52.395354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:52.396304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:52.396304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:52.396304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:52.396304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 631/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:52.401429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.502567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78e95d-441a-4a0c-8ca6-bf0ac5fd0154 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:52.536695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:52.537317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:52.537593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:52.537861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:52.538488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:52.538825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:52.540176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:52.551994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 639/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:52.558442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.592946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:52.662291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc862752-a32a-4a35-a601-056ce966e174 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:52.696520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:52.697206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:52.697488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:52.697761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:52.704315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:52.705840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:10:52.716634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:52.717858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:52.718961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 632/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:52.723735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.782277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:52.824565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a75588-2033-487d-8891-86e710ccda60 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:52.857752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:52.858509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:52.859296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:52.859564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:52.860244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:52.860543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:52.861972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b5db-ab46-471a-a290-a4bc68a583c0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:52.874315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 640/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:52.880515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:52.970790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:52.979257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7ade8-42a4-445c-89b2-2b55e3cbc701 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:53.012565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0b6e5-c247-4b42-a7fb-b1e5f2973d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:53.013241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0b6e5-c247-4b42-a7fb-b1e5f2973d4d None admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:53.013521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0b6e5-c247-4b42-a7fb-b1e5f2973d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:53.013790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0b6e5-c247-4b42-a7fb-b1e5f2973d4d None admin] PATH_INFO: `/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/8c82e23142a343b096a493e01071a506/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:53.015646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0b6e5-c247-4b42-a7fb-b1e5f2973d4d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc243b143dab48d7a19359dc314c43c2, user_id=8c82e23142a343b096a493e01071a506, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:53.017089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0b6e5-c247-4b42-a7fb-b1e5f2973d4d None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:53.152983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 633/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 10 13:10:52 2022] PUT /identity/v3/projects/dc243b143dab48d7a19359dc314c43c2/users/8c82e23142a343b096a493e01071a506/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:10:55.183568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273212c-1745-4b7f-a3fd-3b08ade9d1be None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.184163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273212c-1745-4b7f-a3fd-3b08ade9d1be None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.184163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273212c-1745-4b7f-a3fd-3b08ade9d1be None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.185479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 641/1274] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:55.190636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.191082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.191185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.208957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:55.210191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:55.276276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 634/1275] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:55.301168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273212c-1745-4b7f-a3fd-3b08ade9d1be None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:55.401436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273212c-1745-4b7f-a3fd-3b08ade9d1be None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:55.420290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:55.436959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:55.437552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.437829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.438095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.447871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:10:55.453032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:55.454146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:55.455183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 642/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 10 13:10:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:55.460173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:55.542869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:55.560577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920547af-a4bd-4ae8-a2e0-da2a5a63fb27 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:55.594560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:55.595198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.595482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.595757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.596413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:55.596716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:55.598099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:55.605995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 635/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 10 13:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:55.612020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:55.709716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0239c-65f3-4e6d-92c6-c6152a2e4fcf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:55.742816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:55.743430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.743704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.743971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.753575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:55.754982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:55.769383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:55.771769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:55.772761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:55.772761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:55.772761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:55.772761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 643/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:55.776257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:55.876123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f788c3-fb98-4a2e-ba40-6a8c69272930 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:55.910684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:55.911288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.911570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.911843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.912479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:55.912778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:55.914170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:55.926264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 636/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:55.932204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:55.964324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:56.031834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb281be-dcc2-4825-a473-6043a071a641 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:56.065761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:56.066379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:56.066653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:56.066921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:56.075801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:56.077272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:10:56.087876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:56.089067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:56.090229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 644/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:56.095212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:56.098780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:56.191949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd1723-daa8-4637-9048-c6e057f1dc80 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:56.225389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:56.225927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:56.226207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:56.226477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:56.227194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:56.227498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:56.228905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:56.246196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 637/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 10 13:10:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:56.251983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:56.340297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:56.349368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57839-e2d1-4b34-9358-2e561d413739 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:56.380937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:56.381554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:56.381832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:56.382100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:56.449783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:56.451270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:56.541793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:56.552023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 645/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 10 13:10:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:10:56.558347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:56.620309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:56.661205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba7a4a-5020-4afe-8661-9f1b29fafca1 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:56.706211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:56.706849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:56.707131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:56.707406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:56.720250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:56.722780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:10:56.740858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:56.742133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:10:56.743064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:10:56.743064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:10:56.743064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:56.743064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 638/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:56.748553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:56.867211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549833d7-3448-4e22-a3d9-c5563e3eaea3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:56.909809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:56.910420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:56.910687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:56.910956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:56.911592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:56.911890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:56.913408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:56.928502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 646/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 10 13:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:56.935951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:57.048879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c4871-c287-4686-949b-9f99f2801341 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:57.063416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:57.084264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:57.084900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:57.085212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:57.085487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:57.094584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:57.096031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:10:57.108917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:57.110042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:10:57.111010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: ERROR keystone.server.flask.application Aug 10 13:10:57.111523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 639/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 10 13:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:10:57.116338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:57.187224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:57.241331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df701-35ea-4fdf-80f4-7ca038b9d4b3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:57.278186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:57.278849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:57.279059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:57.279332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:57.280289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:57.280593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:57.282239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07995785-1254-4d7e-96a1-c3441e83f082 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:57.296935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 647/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 10 13:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:10:57.303115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:57.410977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945b782-54bc-4527-8fe9-fc8175d8165a None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:57.444893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:57.445579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:57.445842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:57.446113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:57.446836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] RBAC: Adding query filter params (role.id=dd76871a868143b482c4982027fdc1f5, scope.project.id=dc243b143dab48d7a19359dc314c43c2, user.id=8c82e23142a343b096a493e01071a506) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:57.447146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:57.448623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b543d7-fbfc-4819-b274-5d154f6a9dad None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:57.454894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 640/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 10 13:10:57 2022] GET /identity/v3/role_assignments?user.id=8c82e23142a343b096a493e01071a506&scope.project.id=dc243b143dab48d7a19359dc314c43c2&role.id=dd76871a868143b482c4982027fdc1f5 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:59.497257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8ae8f-cdf3-4ee0-8b14-a3e287c7a354 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:59.497812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8ae8f-cdf3-4ee0-8b14-a3e287c7a354 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:59.497812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8ae8f-cdf3-4ee0-8b14-a3e287c7a354 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:59.499275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 648/1288] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:10:59.504364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:59.504815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:59.504906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:59.523141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:10:59.524314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:59.553875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:59.589778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 641/1289] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:10:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:10:59.612235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8ae8f-cdf3-4ee0-8b14-a3e287c7a354 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:59.714638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8ae8f-cdf3-4ee0-8b14-a3e287c7a354 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:59.725535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:59.748239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:59.748799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:59.749103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:59.749372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:59.749945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:59.751336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:10:59.759716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:10:59.761461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:10:59.762626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 649/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:10:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:10:59.766968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:59.864568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa484a-57a1-4066-9620-6f2f29433130 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:59.897389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:59.897993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:59.898273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:59.898545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:59.899171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:10:59.899468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:59.900821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d1a6e-e72e-4b5a-8f09-ce5d77e38ccb None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:59.906357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 642/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 10 13:10:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:10:59.912229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:00.012217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a51c2-8ce4-4990-80f8-f906f6de831a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:00.044698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:00.045324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:00.045599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:00.048502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:00.048502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:11:00.048502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:00.048502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaffeb-781d-48b0-bce8-7233fa1cece0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:00.052333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 650/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 10 13:10:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:11:02.034335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1503c-353d-4b8e-8037-bce89ea79322 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:02.034912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1503c-353d-4b8e-8037-bce89ea79322 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:02.034912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1503c-353d-4b8e-8037-bce89ea79322 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:02.036227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 643/1293] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:11:02.041453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7946ac-a792-4cef-a231-d9873135f8d8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:02.060764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7946ac-a792-4cef-a231-d9873135f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:02.060764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7946ac-a792-4cef-a231-d9873135f8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:02.060764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-de7946ac-a792-4cef-a231-d9873135f8d8 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:11:02.061414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-de7946ac-a792-4cef-a231-d9873135f8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:11:02.090222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:02.126635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 651/1294] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:11:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:11:02.150207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1503c-353d-4b8e-8037-bce89ea79322 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:02.181735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:02.252133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1503c-353d-4b8e-8037-bce89ea79322 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:02.286147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8919ff-6e07-4a15-8ec0-3512885229d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:02.286680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8919ff-6e07-4a15-8ec0-3512885229d1 None admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:02.286958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8919ff-6e07-4a15-8ec0-3512885229d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:02.287229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8919ff-6e07-4a15-8ec0-3512885229d1 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:02.287958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8919ff-6e07-4a15-8ec0-3512885229d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:02.289335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8919ff-6e07-4a15-8ec0-3512885229d1 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:02.369191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 644/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 10 13:11:02 2022] POST /identity/v3/services => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:11:04.403640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e35f4-7744-41c8-8144-591a443bb4f6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:04.404189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e35f4-7744-41c8-8144-591a443bb4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:04.404189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e35f4-7744-41c8-8144-591a443bb4f6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:04.405846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 652/1296] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:11:04.410998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:04.411398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:04.411633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:04.430024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:11:04.431206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:11:04.503723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 645/1297] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:11:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:11:04.526161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e35f4-7744-41c8-8144-591a443bb4f6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:04.625797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e35f4-7744-41c8-8144-591a443bb4f6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:04.662205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:04.662845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:04.663124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:04.663395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:04.663975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:04.665373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:11:04.674459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:11:04.675478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:11:04.676491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 653/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:11:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:11:04.681744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:04.700693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:04.778644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e4bad-49e7-406c-a9fd-b62d56d9017b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:04.813814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:04.814417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:04.814695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:04.814966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:04.815588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:11:04.815887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:04.817267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:04.822232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 646/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 10 13:11:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:11:04.828556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:04.860234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:04.928228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5696-d29a-469e-98fd-043226f294e9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:04.960495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:04.961120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:04.961401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:04.961673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:04.962368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] RBAC: Adding query filter params (interface=public, service_id=ba7afd85ea884cc7812d23eeda60eba3, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:11:04.962675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:04.964588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1ff6-a152-419f-b4ab-cc8e8487b536 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:04.969556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 654/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 10 13:11:04 2022] GET /identity/v3/endpoints?service_id=ba7afd85ea884cc7812d23eeda60eba3&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:11:04.975321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:05.054679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:05.072186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befbcd3-e190-4fd7-9229-48b2e8ef5a32 None admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:05.105996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9653ea8-d928-4687-97bb-3c3b6725cd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:05.106583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653ea8-d928-4687-97bb-3c3b6725cd3c None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:05.106865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653ea8-d928-4687-97bb-3c3b6725cd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:05.107138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653ea8-d928-4687-97bb-3c3b6725cd3c None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:05.107744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9653ea8-d928-4687-97bb-3c3b6725cd3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:05.109130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9653ea8-d928-4687-97bb-3c3b6725cd3c None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:05.115769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 647/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 10 13:11:04 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:11:07.129270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50019-4e46-4320-b27e-35a5d5a8df3d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:07.129822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50019-4e46-4320-b27e-35a5d5a8df3d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:07.129822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50019-4e46-4320-b27e-35a5d5a8df3d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:07.131192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 655/1302] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:11:07.137443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:07.137865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:07.138014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:07.157162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:11:07.158368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:11:07.225545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 648/1303] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:11:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:11:07.248564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50019-4e46-4320-b27e-35a5d5a8df3d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:07.341144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:07.347546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50019-4e46-4320-b27e-35a5d5a8df3d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:07.379876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:07.380440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:07.380720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:07.381014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:07.381595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:07.382975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:11:07.392153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:11:07.393318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Aug 10 13:11:07.394354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 656/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 10 13:11:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:11:07.399495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:07.474538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:07.497791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b9e35-b4b6-45d6-a894-e366b6e65abd None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:07.532770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:07.533415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:07.533738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:07.534008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] PATH_INFO: `/v3/services` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:07.534635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:11:07.534936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:07.536288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218a84-6e56-41aa-af24-b0e1626bb6cc None admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:07.541305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 649/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 10 13:11:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:11:07.548865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:11:07.649514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6210a-58bc-48c6-b8d5-97972172cfc0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:11:07.683670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f6b17-17fc-4283-9545-e47c41f4ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:11:07.684290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f6b17-17fc-4283-9545-e47c41f4ba0a None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:11:07.684566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f6b17-17fc-4283-9545-e47c41f4ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:11:07.684835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f6b17-17fc-4283-9545-e47c41f4ba0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:11:07.685517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f6b17-17fc-4283-9545-e47c41f4ba0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:11:07.686854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f6b17-17fc-4283-9545-e47c41f4ba0a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:11:07.802989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 657/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:11:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:11:17.569443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=30 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:11:17.798389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:15.586953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b28f9-e7b6-4249-991d-8b2bd2d2e6fa None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:15.587516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b28f9-e7b6-4249-991d-8b2bd2d2e6fa None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:15.587516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b28f9-e7b6-4249-991d-8b2bd2d2e6fa None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:15.595236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8008aa6-bbff-4716-9414-c77e30fe179f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:15.595711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8008aa6-bbff-4716-9414-c77e30fe179f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:15.595794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8008aa6-bbff-4716-9414-c77e30fe179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:15.598723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 650/1307] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:15.698931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d8008aa6-bbff-4716-9414-c77e30fe179f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:15.700169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d8008aa6-bbff-4716-9414-c77e30fe179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:15.877659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 658/1308] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:15.918382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506af6f3-9170-40d5-abd8-4142b3d9fc98 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:15.918919 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506af6f3-9170-40d5-abd8-4142b3d9fc98 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:15.919369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506af6f3-9170-40d5-abd8-4142b3d9fc98 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:15.921589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 651/1309] 10.222.0.40 () {54 vars in 965 bytes} [Wed Aug 10 13:17:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:15.930519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c127d-8a81-428d-8f70-595bd35f4ce1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:15.931102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c127d-8a81-428d-8f70-595bd35f4ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:15.931555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c127d-8a81-428d-8f70-595bd35f4ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:16.055776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ec4c127d-8a81-428d-8f70-595bd35f4ce1 None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:16.057061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ec4c127d-8a81-428d-8f70-595bd35f4ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:16.080081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:16.141722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 659/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Wed Aug 10 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:16.150803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506af6f3-9170-40d5-abd8-4142b3d9fc98 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:16.152615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-506af6f3-9170-40d5-abd8-4142b3d9fc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:16.264062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506af6f3-9170-40d5-abd8-4142b3d9fc98 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:16.341321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:16.341856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:16.342138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:16.342410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:16.344582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:16.402827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:16.466800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:16.468256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25444402-dcb9-4ce8-9b1a-a50107bb074d service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:16.490067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 652/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:18.897437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0cfcf-658c-4fd9-a915-91f194293164 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:18.897965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0cfcf-658c-4fd9-a915-91f194293164 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:18.897965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0cfcf-658c-4fd9-a915-91f194293164 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:18.899513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 660/1312] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:18.905197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1781-15e9-400a-95fb-8c766dcbc48e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:18.905709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1781-15e9-400a-95fb-8c766dcbc48e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:18.905788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1781-15e9-400a-95fb-8c766dcbc48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:18.991603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-201b1781-15e9-400a-95fb-8c766dcbc48e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:18.993139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-201b1781-15e9-400a-95fb-8c766dcbc48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:19.106113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 653/1313] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:19.146532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0cfcf-658c-4fd9-a915-91f194293164 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:19.148175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-46c0cfcf-658c-4fd9-a915-91f194293164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:19.260251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0cfcf-658c-4fd9-a915-91f194293164 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:19.346503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:19.348862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:19.349504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:19.350152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:19.350431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:19.352043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:19.461005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:19.462477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1695d8d-26e3-4308-aae8-60feaa377a7f service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:19.480644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 661/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:21.981558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cba94-dcd0-4f83-bc4b-e67b3a53acf7 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:21.982062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cba94-dcd0-4f83-bc4b-e67b3a53acf7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:21.982062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cba94-dcd0-4f83-bc4b-e67b3a53acf7 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:21.983354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 654/1315] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:21.989514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b848715-3eec-4bab-b78f-d05f3d464a9b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:21.989920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b848715-3eec-4bab-b78f-d05f3d464a9b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:21.990064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b848715-3eec-4bab-b78f-d05f3d464a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:22.054515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4b848715-3eec-4bab-b78f-d05f3d464a9b None None] MFA Rules not processed for user `a8045692cd1e4d48be5039398bca0cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:22.055708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4b848715-3eec-4bab-b78f-d05f3d464a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:22.106292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:22.148877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 662/1316] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:17:22.190179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cba94-dcd0-4f83-bc4b-e67b3a53acf7 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:22.191714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-813cba94-dcd0-4f83-bc4b-e67b3a53acf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:22.201180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:22.292618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cba94-dcd0-4f83-bc4b-e67b3a53acf7 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:22.364004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:22.364611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:22.364891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:22.365203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:22.366788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:22.427276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:22.473221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:22.474655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9933b2-2751-4631-a703-bb01c1875614 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:22.494794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 655/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:25.410114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9b87-1b8c-4a1a-927c-1b15f4196eaf None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.410709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9b87-1b8c-4a1a-927c-1b15f4196eaf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.410709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9b87-1b8c-4a1a-927c-1b15f4196eaf None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.412070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 663/1318] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:25.417259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50b115-9c34-42e3-9094-99e0f2f81ef1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.417702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50b115-9c34-42e3-9094-99e0f2f81ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.417816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50b115-9c34-42e3-9094-99e0f2f81ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.481423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ac50b115-9c34-42e3-9094-99e0f2f81ef1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:25.482609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ac50b115-9c34-42e3-9094-99e0f2f81ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:25.576257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 656/1319] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:25.615619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a9b87-1b8c-4a1a-927c-1b15f4196eaf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.617193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-053a9b87-1b8c-4a1a-927c-1b15f4196eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:25.720582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a9b87-1b8c-4a1a-927c-1b15f4196eaf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.792557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.793186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.793470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.793739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.795335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:25.802624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:25.901954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.903304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c42a8-d4fb-49e3-af32-8e03eaf3d666 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.921127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 664/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:28.937851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8714bec-e8bd-4bb2-8fd0-5b6c02512f76 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:28.938515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8714bec-e8bd-4bb2-8fd0-5b6c02512f76 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:28.938515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8714bec-e8bd-4bb2-8fd0-5b6c02512f76 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:28.939670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 657/1321] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:28.945034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5b45-248b-4da3-bdd3-d76038d0b022 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:28.945772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5b45-248b-4da3-bdd3-d76038d0b022 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:28.946049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5b45-248b-4da3-bdd3-d76038d0b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:29.013468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d88d5b45-248b-4da3-bdd3-d76038d0b022 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:29.014663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d88d5b45-248b-4da3-bdd3-d76038d0b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:29.076955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:29.122760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 665/1322] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:29.159942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8714bec-e8bd-4bb2-8fd0-5b6c02512f76 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:29.161513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e8714bec-e8bd-4bb2-8fd0-5b6c02512f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:29.181968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:29.271656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8714bec-e8bd-4bb2-8fd0-5b6c02512f76 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:29.346029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:29.346650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:29.346934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:29.347205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:29.348819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:29.428483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:29.466981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:29.468447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c577186-9c05-4fec-8d4d-58805ee5d9d6 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:29.487711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 658/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:31.569085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe6059-6bd9-4941-b582-49afbbf6e811 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.569622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe6059-6bd9-4941-b582-49afbbf6e811 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.569622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe6059-6bd9-4941-b582-49afbbf6e811 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.570933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 666/1324] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:31.576210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946898-4c2c-46d1-b6df-dde74669e8cf None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.576630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946898-4c2c-46d1-b6df-dde74669e8cf None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.576771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946898-4c2c-46d1-b6df-dde74669e8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.643788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-74946898-4c2c-46d1-b6df-dde74669e8cf None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:31.644984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-74946898-4c2c-46d1-b6df-dde74669e8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:31.742136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 659/1325] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:31.793815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a04a5f-6d9f-4edf-87a1-1377b2674302 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.794191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a04a5f-6d9f-4edf-87a1-1377b2674302 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.794531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a04a5f-6d9f-4edf-87a1-1377b2674302 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.795949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 667/1326] 10.222.0.40 () {54 vars in 965 bytes} [Wed Aug 10 13:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:31.801955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4eeae-5a3f-4a7c-9ca3-a783e2394b49 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.802240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4eeae-5a3f-4a7c-9ca3-a783e2394b49 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.802507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4eeae-5a3f-4a7c-9ca3-a783e2394b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.833838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:31.877986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b1a4eeae-5a3f-4a7c-9ca3-a783e2394b49 None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:31.879240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b1a4eeae-5a3f-4a7c-9ca3-a783e2394b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:31.956277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 660/1327] 10.222.0.40 () {58 vars in 1062 bytes} [Wed Aug 10 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:17:31.965212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a04a5f-6d9f-4edf-87a1-1377b2674302 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:31.966742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-20a04a5f-6d9f-4edf-87a1-1377b2674302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:32.079443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a04a5f-6d9f-4edf-87a1-1377b2674302 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:32.155524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:32.156690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:32.156690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:32.157287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:32.158758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:32.176851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:32.277726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:32.279443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a0db-c491-45db-9e29-3da0ed361fc3 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:32.298738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 668/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:34.623585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbec227-c61e-403f-94ca-ae05de83c79e None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:34.624111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbec227-c61e-403f-94ca-ae05de83c79e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:34.624111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbec227-c61e-403f-94ca-ae05de83c79e None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:34.626200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 661/1329] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:34.631416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3f28-af4d-4bfe-8ba4-d31f28f0aa4b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:34.631805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3f28-af4d-4bfe-8ba4-d31f28f0aa4b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:34.631966 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3f28-af4d-4bfe-8ba4-d31f28f0aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:34.701189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-356b3f28-af4d-4bfe-8ba4-d31f28f0aa4b None None] MFA Rules not processed for user `a8045692cd1e4d48be5039398bca0cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:34.702414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-356b3f28-af4d-4bfe-8ba4-d31f28f0aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:34.767651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:34.799220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 669/1330] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:34.847155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbec227-c61e-403f-94ca-ae05de83c79e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:34.848699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dbbec227-c61e-403f-94ca-ae05de83c79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:34.959058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbec227-c61e-403f-94ca-ae05de83c79e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:34.978852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:35.033569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:35.034174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:35.034451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:35.034718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:35.036304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:35.147503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:35.148824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c42b2f-1f48-4266-800c-e3569a667bbb service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:35.167371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 662/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:37.878049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b08c-28d6-4577-989c-22d237d7f60a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:37.878649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b08c-28d6-4577-989c-22d237d7f60a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:37.878649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b08c-28d6-4577-989c-22d237d7f60a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:37.879938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 670/1332] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:37.884979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947470-05ec-49f2-b2a0-7cd7c0cf5a3e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:37.885380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947470-05ec-49f2-b2a0-7cd7c0cf5a3e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:37.885556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947470-05ec-49f2-b2a0-7cd7c0cf5a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:37.947388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:37.950218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-97947470-05ec-49f2-b2a0-7cd7c0cf5a3e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:37.951450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-97947470-05ec-49f2-b2a0-7cd7c0cf5a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:38.050201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 663/1333] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:38.086864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b08c-28d6-4577-989c-22d237d7f60a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:38.088385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fd50b08c-28d6-4577-989c-22d237d7f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:38.193041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b08c-28d6-4577-989c-22d237d7f60a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:38.265577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:38.266261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:38.266538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:38.266808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:38.268401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:38.294709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:38.376639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:38.378097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea765401-be2f-4e41-80d0-d5774a92be62 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:38.395960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 671/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:40.477348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d1cc1-da81-4fbe-9ce6-148485d15b5e None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:40.477914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d1cc1-da81-4fbe-9ce6-148485d15b5e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:40.477914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d1cc1-da81-4fbe-9ce6-148485d15b5e None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:40.479288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 664/1335] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:40.484907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc04598-9afd-4ce4-9ee9-f01718abb25e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:40.485367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc04598-9afd-4ce4-9ee9-f01718abb25e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:40.485480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc04598-9afd-4ce4-9ee9-f01718abb25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:40.549565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7bc04598-9afd-4ce4-9ee9-f01718abb25e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:40.550771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7bc04598-9afd-4ce4-9ee9-f01718abb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:40.622465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:40.645360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 672/1336] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:17:40.686706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d1cc1-da81-4fbe-9ce6-148485d15b5e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:40.688303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3d8d1cc1-da81-4fbe-9ce6-148485d15b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:40.790630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d1cc1-da81-4fbe-9ce6-148485d15b5e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:40.819391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:40.864603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:40.865243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:40.865512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:40.865831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:40.867407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:40.976068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:40.977532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51fd64-8752-4c64-bc6c-a6a681ee4d95 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:40.995943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 665/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:43.739516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367345-4fa9-4af0-8c20-bf92ddbcbf98 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:43.740071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367345-4fa9-4af0-8c20-bf92ddbcbf98 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:43.740071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367345-4fa9-4af0-8c20-bf92ddbcbf98 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:43.741520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 673/1338] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:43.746669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfe30f-e5fb-46f8-8297-6f7572af3634 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:43.747183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfe30f-e5fb-46f8-8297-6f7572af3634 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:43.747289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfe30f-e5fb-46f8-8297-6f7572af3634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:43.814694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2abfe30f-e5fb-46f8-8297-6f7572af3634 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:43.815918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2abfe30f-e5fb-46f8-8297-6f7572af3634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:43.824027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:43.911757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 666/1339] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:43.948826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367345-4fa9-4af0-8c20-bf92ddbcbf98 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:43.950444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-94367345-4fa9-4af0-8c20-bf92ddbcbf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:44.055121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367345-4fa9-4af0-8c20-bf92ddbcbf98 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:44.128503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:44.129127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:44.129403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:44.129670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:44.131239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:44.166876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:44.239717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:44.241449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c039e-211c-4381-8d60-b2536c72106d service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:44.261088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 674/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:48.600210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aa530-a438-4bc2-9af1-7ef70937882d None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:48.600895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aa530-a438-4bc2-9af1-7ef70937882d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:48.600895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aa530-a438-4bc2-9af1-7ef70937882d None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:48.602135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 667/1341] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:48.607352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcae0c-4128-4b67-b1c7-3785bdd67607 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:48.607818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcae0c-4128-4b67-b1c7-3785bdd67607 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:48.607912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcae0c-4128-4b67-b1c7-3785bdd67607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:48.678423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-94bcae0c-4128-4b67-b1c7-3785bdd67607 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:48.679634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-94bcae0c-4128-4b67-b1c7-3785bdd67607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:48.764040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:48.775792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 675/1342] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:48.815265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050aa530-a438-4bc2-9af1-7ef70937882d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:48.816793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-050aa530-a438-4bc2-9af1-7ef70937882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:48.922900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050aa530-a438-4bc2-9af1-7ef70937882d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:48.960575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:48.997250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:48.997858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:48.998136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:48.998406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:48.999987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:49.108434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:49.110055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd68796-e326-4f8c-9b4e-34c07ed1cc5e service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:49.130740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 668/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:52.106692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085241-2c82-49de-83b0-ff2ef502b1de None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:52.107223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085241-2c82-49de-83b0-ff2ef502b1de None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:52.107223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085241-2c82-49de-83b0-ff2ef502b1de None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:52.109316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 676/1344] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:52.114339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52a544-bd71-4401-a4ae-f854ca508800 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:52.114767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52a544-bd71-4401-a4ae-f854ca508800 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:52.114900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52a544-bd71-4401-a4ae-f854ca508800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:52.181626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7f52a544-bd71-4401-a4ae-f854ca508800 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:52.182822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7f52a544-bd71-4401-a4ae-f854ca508800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:52.200728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:52.279055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 669/1345] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:52.317268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71085241-2c82-49de-83b0-ff2ef502b1de None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:52.318810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-71085241-2c82-49de-83b0-ff2ef502b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:52.421949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71085241-2c82-49de-83b0-ff2ef502b1de None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:52.497721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:52.498377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:52.498665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:52.498962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:52.500596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:52.547063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:52.615998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:52.617526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ecef8-e8a0-41a1-b263-0dc745e8a915 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:52.639510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 677/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:57.170957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc44a-52cd-4cdd-94f5-4dcf51829e14 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:57.171699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc44a-52cd-4cdd-94f5-4dcf51829e14 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:57.171699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc44a-52cd-4cdd-94f5-4dcf51829e14 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:57.172958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 670/1347] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:17:57.178482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e51bcb-a116-45f3-8bb3-dd5898dad7ce None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:57.178979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e51bcb-a116-45f3-8bb3-dd5898dad7ce None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:57.179081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e51bcb-a116-45f3-8bb3-dd5898dad7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:57.252358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e8e51bcb-a116-45f3-8bb3-dd5898dad7ce None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:57.253603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e8e51bcb-a116-45f3-8bb3-dd5898dad7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:57.371226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:57.377645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 678/1348] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:57.416670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebc44a-52cd-4cdd-94f5-4dcf51829e14 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:57.418259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-47ebc44a-52cd-4cdd-94f5-4dcf51829e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:57.539476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebc44a-52cd-4cdd-94f5-4dcf51829e14 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:57.590826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:57.618637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:57.619262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:57.619551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:57.619824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:57.621487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:57.756588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:57.758059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d02c-e859-4233-9e2b-167341252612 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:57.778048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 671/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:02.789861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed1350-baef-42a0-b178-1e024e8c2b21 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:02.795126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed1350-baef-42a0-b178-1e024e8c2b21 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:02.795420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed1350-baef-42a0-b178-1e024e8c2b21 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:02.797224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 679/1350] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:18:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:02.802760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46a268-2c02-459e-a1dd-bbccae5cd11f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:02.803192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46a268-2c02-459e-a1dd-bbccae5cd11f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:02.803337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46a268-2c02-459e-a1dd-bbccae5cd11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:02.921631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0e46a268-2c02-459e-a1dd-bbccae5cd11f None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:02.928404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0e46a268-2c02-459e-a1dd-bbccae5cd11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:02.943187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:03.043040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 672/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:07.254725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb3fd-2bf4-40db-b146-4232916a3f0e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:07.255568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb3fd-2bf4-40db-b146-4232916a3f0e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:07.255992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb3fd-2bf4-40db-b146-4232916a3f0e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:07.257556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 680/1352] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:18:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:07.262531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b5fd4-f0b5-4660-9547-597d39ebfff0 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:07.263433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b5fd4-f0b5-4660-9547-597d39ebfff0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:07.263906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b5fd4-f0b5-4660-9547-597d39ebfff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:07.350561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b73b5fd4-f0b5-4660-9547-597d39ebfff0 None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:07.352016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b73b5fd4-f0b5-4660-9547-597d39ebfff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:07.431370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:07.478907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 673/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:18:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:08.796682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eacfd-dbf8-48cc-8baa-7648d8012f8b None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:08.797260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eacfd-dbf8-48cc-8baa-7648d8012f8b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:08.797260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eacfd-dbf8-48cc-8baa-7648d8012f8b None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:08.798598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 681/1354] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:08.803834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1ee7-12c2-422a-a549-3c69de821470 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:08.804247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1ee7-12c2-422a-a549-3c69de821470 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:08.804412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1ee7-12c2-422a-a549-3c69de821470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:08.879202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-55db1ee7-12c2-422a-a549-3c69de821470 None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:08.880415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-55db1ee7-12c2-422a-a549-3c69de821470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:08.990365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 674/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:18:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:12.371480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff5c01-c1f2-48c9-afba-b3008e443d6f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:12.372455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff5c01-c1f2-48c9-afba-b3008e443d6f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:12.373370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff5c01-c1f2-48c9-afba-b3008e443d6f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:12.376198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 682/1356] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:12.382359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f06efc-447c-4a95-b1de-4cd9633e6e0c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:12.382835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f06efc-447c-4a95-b1de-4cd9633e6e0c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:12.382962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f06efc-447c-4a95-b1de-4cd9633e6e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:12.434282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:12.476633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-30f06efc-447c-4a95-b1de-4cd9633e6e0c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:12.478246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-30f06efc-447c-4a95-b1de-4cd9633e6e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:12.586035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 675/1357] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:12.647094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21ba8-17cf-4a7c-a77f-d824b6fd3799 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:12.648471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21ba8-17cf-4a7c-a77f-d824b6fd3799 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:12.648889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21ba8-17cf-4a7c-a77f-d824b6fd3799 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:12.661396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 683/1358] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 10 13:18:12 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:12.661525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9701d-0ee5-4525-9624-33560723a94e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:12.661865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9701d-0ee5-4525-9624-33560723a94e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:12.662140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9701d-0ee5-4525-9624-33560723a94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:12.754798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-27c9701d-0ee5-4525-9624-33560723a94e None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:12.756512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-27c9701d-0ee5-4525-9624-33560723a94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:12.818890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:12.890685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 676/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 10 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:12.898779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e21ba8-17cf-4a7c-a77f-d824b6fd3799 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:12.900318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-62e21ba8-17cf-4a7c-a77f-d824b6fd3799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:13.053558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e21ba8-17cf-4a7c-a77f-d824b6fd3799 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:13.144831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:13.189045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:13.189674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:13.189958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:13.190237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:13.191873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:13.292486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:13.294268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f9442-e678-44b5-95a5-165b8761a68d service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:13.315420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 684/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:15.548820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4d967-32f9-402c-a766-4a8a8e49f440 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.549402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4d967-32f9-402c-a766-4a8a8e49f440 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.549663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4d967-32f9-402c-a766-4a8a8e49f440 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.551431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 677/1361] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:15.557092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4367-551a-45a8-bbb2-6ff4fdf08c73 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.557638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4367-551a-45a8-bbb2-6ff4fdf08c73 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.557638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4367-551a-45a8-bbb2-6ff4fdf08c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.627784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-eccd4367-551a-45a8-bbb2-6ff4fdf08c73 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:15.629123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eccd4367-551a-45a8-bbb2-6ff4fdf08c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:15.676841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:15.754772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 685/1362] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:15.800750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4d967-32f9-402c-a766-4a8a8e49f440 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.802382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-baf4d967-32f9-402c-a766-4a8a8e49f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:15.978738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4d967-32f9-402c-a766-4a8a8e49f440 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.989242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:16.117759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.118229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.118575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.119132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.120839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:16.222477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.224529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a7aa5-03c0-4a5b-b850-b09c776bc500 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.244616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 678/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:18.706044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc8de-701c-4bb3-a622-75896d3f77d9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:18.706585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc8de-701c-4bb3-a622-75896d3f77d9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:18.706585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc8de-701c-4bb3-a622-75896d3f77d9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:18.708204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 686/1364] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:18:18.713431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54974-37b1-42b9-b762-ce4787d68256 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:18.720346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54974-37b1-42b9-b762-ce4787d68256 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:18.720346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54974-37b1-42b9-b762-ce4787d68256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:18.730661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:18.794843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b1d54974-37b1-42b9-b762-ce4787d68256 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:18.796492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b1d54974-37b1-42b9-b762-ce4787d68256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:18.913131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 679/1365] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:18.959896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5223509-241a-430e-9fe3-d3967cc744bc None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:18.960309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5223509-241a-430e-9fe3-d3967cc744bc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:18.960551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5223509-241a-430e-9fe3-d3967cc744bc None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:18.961958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 687/1366] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 10 13:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:18:18.968485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7360-ebcb-4fe4-b0c0-1497e410b694 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:18.968780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7360-ebcb-4fe4-b0c0-1497e410b694 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:18.973017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7360-ebcb-4fe4-b0c0-1497e410b694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:19.052763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f98b7360-ebcb-4fe4-b0c0-1497e410b694 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:19.054012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f98b7360-ebcb-4fe4-b0c0-1497e410b694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:19.094097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:19.185067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 680/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 10 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:19.196421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5223509-241a-430e-9fe3-d3967cc744bc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:19.198392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b5223509-241a-430e-9fe3-d3967cc744bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:19.360022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5223509-241a-430e-9fe3-d3967cc744bc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:19.381599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:19.481130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:19.481642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:19.481918 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:19.482203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:19.484346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:19.579692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:19.581305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c9c6b-a475-41cf-856e-1c825d56c221 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:19.600509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 688/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:20.791422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a44d77-94da-4523-ab10-e7948eaa7975 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:20.792013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a44d77-94da-4523-ab10-e7948eaa7975 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:20.792013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a44d77-94da-4523-ab10-e7948eaa7975 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:20.797100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 681/1369] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:20.799036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef370143-cbc3-4848-b1b7-4adb60013e2d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:20.799470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef370143-cbc3-4848-b1b7-4adb60013e2d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:20.799567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef370143-cbc3-4848-b1b7-4adb60013e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:20.831793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:20.867329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ef370143-cbc3-4848-b1b7-4adb60013e2d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:20.868552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef370143-cbc3-4848-b1b7-4adb60013e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:20.979773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 689/1370] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:21.321275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cda835-88a3-4c25-a041-86f6d89ce448 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.321686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cda835-88a3-4c25-a041-86f6d89ce448 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.321875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cda835-88a3-4c25-a041-86f6d89ce448 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.323424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 682/1371] 10.222.0.40 () {54 vars in 964 bytes} [Wed Aug 10 13:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:21.329823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c3967-e0cf-446a-9d71-3e0e0536b24a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.329959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c3967-e0cf-446a-9d71-3e0e0536b24a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.330157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c3967-e0cf-446a-9d71-3e0e0536b24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.409282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cf9c3967-e0cf-446a-9d71-3e0e0536b24a None None] MFA Rules not processed for user `64ff5fa2e326485bb2f915fa4f8870fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:21.410522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cf9c3967-e0cf-446a-9d71-3e0e0536b24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:21.460464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:21.494852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 690/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Aug 10 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:21.502770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cda835-88a3-4c25-a041-86f6d89ce448 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:21.504275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-00cda835-88a3-4c25-a041-86f6d89ce448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:21.619860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cda835-88a3-4c25-a041-86f6d89ce448 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:21.669282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:21.704212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:21.704826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.705134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.705718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.707060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:21.827858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:21.829325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40296e4-1769-4fab-bb57-44a7b97a1140 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:21.848336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 683/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:21.916115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e229-29b1-47c2-be37-3c22af005cd1 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.916669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e229-29b1-47c2-be37-3c22af005cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.916669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e229-29b1-47c2-be37-3c22af005cd1 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.918150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 691/1374] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:21.924654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f587-5e78-427d-95cd-4adf1f3f865e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.924928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f587-5e78-427d-95cd-4adf1f3f865e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.925536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f587-5e78-427d-95cd-4adf1f3f865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.017613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4b87f587-5e78-427d-95cd-4adf1f3f865e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:22.018877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4b87f587-5e78-427d-95cd-4adf1f3f865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:22.046452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:22.143580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 684/1375] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:22.180485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e229-29b1-47c2-be37-3c22af005cd1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.182059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-05f0e229-29b1-47c2-be37-3c22af005cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:22.334343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e229-29b1-47c2-be37-3c22af005cd1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.386803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:22.448679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.449316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.449592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.449862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.452086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:22.542800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.547695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e785-efa7-4d2b-8d1e-a4b2b6e8b660 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.562720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 692/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:24.837072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290e47-34ec-4ec9-a9f2-074a99e0acdd None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:24.837608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290e47-34ec-4ec9-a9f2-074a99e0acdd None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:24.837711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290e47-34ec-4ec9-a9f2-074a99e0acdd None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:24.838988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 685/1377] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:24.844461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9b6e5-e3dc-4c98-81d1-cc519dc8b4ec None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:24.844893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9b6e5-e3dc-4c98-81d1-cc519dc8b4ec None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:24.845073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9b6e5-e3dc-4c98-81d1-cc519dc8b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:24.913457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-43c9b6e5-e3dc-4c98-81d1-cc519dc8b4ec None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:24.914587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-43c9b6e5-e3dc-4c98-81d1-cc519dc8b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:24.921743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:25.031041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 693/1378] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:25.082720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290e47-34ec-4ec9-a9f2-074a99e0acdd None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:25.084201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-86290e47-34ec-4ec9-a9f2-074a99e0acdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:25.231142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290e47-34ec-4ec9-a9f2-074a99e0acdd None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:25.234968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:25.349394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:25.350032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:25.350311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:25.350581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:25.352174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:25.470158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:25.471958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efbcd8-ca07-4001-ae92-5f53d82d6399 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:25.494543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 686/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:27.727174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a07d8-9c30-43e8-a1f2-00c575e4dfda None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:27.727752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a07d8-9c30-43e8-a1f2-00c575e4dfda None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:27.727752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a07d8-9c30-43e8-a1f2-00c575e4dfda None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:27.729296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 694/1380] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:27.734384 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea87226-c398-4fa0-b70a-838c33deba11 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:27.734850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea87226-c398-4fa0-b70a-838c33deba11 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:27.735216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea87226-c398-4fa0-b70a-838c33deba11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:27.749407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:27.817291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-aea87226-c398-4fa0-b70a-838c33deba11 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:27.818466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-aea87226-c398-4fa0-b70a-838c33deba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:27.924870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 687/1381] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:27.970231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a07d8-9c30-43e8-a1f2-00c575e4dfda None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:27.971741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6a2a07d8-9c30-43e8-a1f2-00c575e4dfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:28.115710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:28.126592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a07d8-9c30-43e8-a1f2-00c575e4dfda None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:28.249181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:28.249749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:28.250027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:28.250296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:28.251881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:28.347213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:28.348802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8b8c2-4513-4002-9bcf-759a2d001c47 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:28.361607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:28.372155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 695/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:30.580978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187cf7b-3b2d-4db6-b497-495f7d944554 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:30.581663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187cf7b-3b2d-4db6-b497-495f7d944554 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:30.581663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187cf7b-3b2d-4db6-b497-495f7d944554 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:30.582897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 688/1383] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:18:30.588010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0b2e-a11e-4cd9-b657-d2379d4a2676 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:30.588453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0b2e-a11e-4cd9-b657-d2379d4a2676 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:30.588555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0b2e-a11e-4cd9-b657-d2379d4a2676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:30.661559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-eb3d0b2e-a11e-4cd9-b657-d2379d4a2676 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:30.662792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eb3d0b2e-a11e-4cd9-b657-d2379d4a2676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:30.765502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 696/1384] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:30.806949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187cf7b-3b2d-4db6-b497-495f7d944554 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:30.808470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1187cf7b-3b2d-4db6-b497-495f7d944554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:30.927625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:31.021920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187cf7b-3b2d-4db6-b497-495f7d944554 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:31.210979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:31.211532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:31.211812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:31.212101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:31.213827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:31.266299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:31.309238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:31.311917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7d313-728b-45c3-91b6-513a3a971ee7 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:31.330654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 689/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:18:33.448289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b65c-eb54-4ec6-b0d3-1032629a87e9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:33.448950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b65c-eb54-4ec6-b0d3-1032629a87e9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:33.448950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b65c-eb54-4ec6-b0d3-1032629a87e9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:33.450289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 697/1386] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:33.456614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f93d2-15de-41e1-8612-85708f87144f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:33.457190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f93d2-15de-41e1-8612-85708f87144f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:33.457289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f93d2-15de-41e1-8612-85708f87144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:33.527322 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-bc5f93d2-15de-41e1-8612-85708f87144f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:33.528640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bc5f93d2-15de-41e1-8612-85708f87144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:33.631289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 690/1387] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:33.679265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b65c-eb54-4ec6-b0d3-1032629a87e9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:33.681069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2291b65c-eb54-4ec6-b0d3-1032629a87e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:33.764158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:33.844654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b65c-eb54-4ec6-b0d3-1032629a87e9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:33.980199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:33.980916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:33.981292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:33.981638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:33.984035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:34.028700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:34.083523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:34.085179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e7277-631f-4047-8cd5-17315565f78a service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:34.103370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 698/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:36.325133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee72ad1-555a-4899-89bb-2bd05de2c28e None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.326268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee72ad1-555a-4899-89bb-2bd05de2c28e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.326268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee72ad1-555a-4899-89bb-2bd05de2c28e None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.327599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 691/1389] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:36.332697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe212d4-4d41-40cf-af1c-40f3ada437f1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.333173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe212d4-4d41-40cf-af1c-40f3ada437f1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.333263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe212d4-4d41-40cf-af1c-40f3ada437f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.403353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2fe212d4-4d41-40cf-af1c-40f3ada437f1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:36.404905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2fe212d4-4d41-40cf-af1c-40f3ada437f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:36.505116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 699/1390] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:36.549872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee72ad1-555a-4899-89bb-2bd05de2c28e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.551442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-aee72ad1-555a-4899-89bb-2bd05de2c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:36.586163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:36.710926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee72ad1-555a-4899-89bb-2bd05de2c28e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.811243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:36.823649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.824259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.824788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.825081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.827100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:36.916336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.917877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b63df-c26a-4d67-a15c-38c7634ce032 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.936862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 692/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:37.316754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f58106-1f05-4b78-95db-89ab0f6e5b64 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.317051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f58106-1f05-4b78-95db-89ab0f6e5b64 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.317333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f58106-1f05-4b78-95db-89ab0f6e5b64 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.318877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 700/1392] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:37.323824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d8e2-6155-465f-aecb-53e983dacb02 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.324120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d8e2-6155-465f-aecb-53e983dacb02 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.324397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d8e2-6155-465f-aecb-53e983dacb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.392713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4499d8e2-6155-465f-aecb-53e983dacb02 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:37.393972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4499d8e2-6155-465f-aecb-53e983dacb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:37.454388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:37.493468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 693/1393] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:37.551081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649d97a-2f4d-4c0f-9e02-44f3959d3983 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.551556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649d97a-2f4d-4c0f-9e02-44f3959d3983 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.551556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649d97a-2f4d-4c0f-9e02-44f3959d3983 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.552930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 701/1394] 10.222.0.40 () {54 vars in 967 bytes} [Wed Aug 10 13:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:37.558240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dd1d4-1bac-440d-b32f-23c59441aa9c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.558532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dd1d4-1bac-440d-b32f-23c59441aa9c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.558810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dd1d4-1bac-440d-b32f-23c59441aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.636934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b82dd1d4-1bac-440d-b32f-23c59441aa9c None None] MFA Rules not processed for user `8279c74cfda94068b5b392123282e556`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:37.638200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b82dd1d4-1bac-440d-b32f-23c59441aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:37.721319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 694/1395] 10.222.0.40 () {58 vars in 1064 bytes} [Wed Aug 10 13:18:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:37.728943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649d97a-2f4d-4c0f-9e02-44f3959d3983 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.730480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2649d97a-2f4d-4c0f-9e02-44f3959d3983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:37.744229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:37.837095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649d97a-2f4d-4c0f-9e02-44f3959d3983 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.911685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.912293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.912572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.912849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.914471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:37.981143 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:38.023056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:38.024499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a1071-e1c9-4f70-a9a6-adc1d4022636 service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:38.043046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 702/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:38.919177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb0f-8e7e-45ea-8523-2704afed4f72 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:38.920159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb0f-8e7e-45ea-8523-2704afed4f72 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:38.920392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb0f-8e7e-45ea-8523-2704afed4f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:38.987941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-dd40eb0f-8e7e-45ea-8523-2704afed4f72 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:38.989264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dd40eb0f-8e7e-45ea-8523-2704afed4f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:38.997627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:39.077199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 695/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:18:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:39.101488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7f28-a72e-4f30-ad0d-6d939f815ef2 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:39.101909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7f28-a72e-4f30-ad0d-6d939f815ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:39.102210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7f28-a72e-4f30-ad0d-6d939f815ef2 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:39.103744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 703/1398] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:39.110135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:39.110426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:39.110698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:39.120306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a82f4-4a19-4618-b9b7-213837d41725 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:39.121060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a82f4-4a19-4618-b9b7-213837d41725 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:39.121336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a82f4-4a19-4618-b9b7-213837d41725 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:39.123485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 704/1399] 10.222.0.40 () {54 vars in 943 bytes} [Wed Aug 10 13:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:18:39.129821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:39.130192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:39.130537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:39.179017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:39.180148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:39.212263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] MFA Rules not processed for user `1bb11e8a7a124630bf9b0e611324fabf`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:39.213516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:39.278347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 696/1400] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:39.320729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 705/1401] 10.222.0.40 () {58 vars in 1040 bytes} [Wed Aug 10 13:18:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:39.322997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:39.324205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:39.327434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:39.330948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:39.335681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:39.341309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0352c3a-80b6-4f28-ad29-5a872eff68c8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:39.441585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee8f8a-72d0-4131-959c-3d32b7928e0f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:39.446311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:39.446924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:39.447204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:39.447473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:39.449178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:39.532967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:39.534608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b88e46-b469-434a-b694-0c6f56ed1e1c service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:39.542491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 706/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:18:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:18:39.554814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:39.557406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:39.557920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:39.558206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:39.558584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:39.560224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:39.667788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:39.669598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef5521-7bf6-4175-b232-f0acd84b0648 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:39.690638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 697/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:42.110777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4b0e-88ed-4408-adda-f50829433eca None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:42.111385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4b0e-88ed-4408-adda-f50829433eca None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:42.111385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4b0e-88ed-4408-adda-f50829433eca None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:42.112833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 707/1404] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:18:42.118117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4ce7-4658-4dd4-839c-4f99df05d087 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:42.118636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4ce7-4658-4dd4-839c-4f99df05d087 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:42.118636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4ce7-4658-4dd4-839c-4f99df05d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:42.193660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2edf4ce7-4658-4dd4-839c-4f99df05d087 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:42.194904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2edf4ce7-4658-4dd4-839c-4f99df05d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:42.273319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:42.311327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 698/1405] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:42.362265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4b0e-88ed-4408-adda-f50829433eca None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:42.363729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a5d4b0e-88ed-4408-adda-f50829433eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:42.402201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:42.531529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4b0e-88ed-4408-adda-f50829433eca None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:42.858783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:42.858783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:42.858783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:42.858783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:42.859137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:42.859137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:42.859137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:42.859137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd72286-bdbd-40c6-8de3-4b201513b799 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:42.859137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 708/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:45.507979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca644260-b495-42fd-ba23-55e3b07e8751 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:45.508559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca644260-b495-42fd-ba23-55e3b07e8751 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:45.508559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca644260-b495-42fd-ba23-55e3b07e8751 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:45.509906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 699/1407] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:18:45.515992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1220b-0a02-44c1-888e-9ab9670b45df None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:45.516420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1220b-0a02-44c1-888e-9ab9670b45df None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:45.516620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1220b-0a02-44c1-888e-9ab9670b45df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:45.591752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e0d1220b-0a02-44c1-888e-9ab9670b45df None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:45.592956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0d1220b-0a02-44c1-888e-9ab9670b45df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:45.665200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:45.704437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 709/1408] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:45.754203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca644260-b495-42fd-ba23-55e3b07e8751 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:45.755730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ca644260-b495-42fd-ba23-55e3b07e8751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:45.920962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca644260-b495-42fd-ba23-55e3b07e8751 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:45.984712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:46.052080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.052683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.052962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.053305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.054909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:46.154893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.156794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37c301-1767-426e-93e0-22a8b76541aa service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.178617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 700/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:55.715102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=14 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:56.176219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=73 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:03.835769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862876a-0736-4f36-bd1f-5b0f98f5f70d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:03.836335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862876a-0736-4f36-bd1f-5b0f98f5f70d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:03.836335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862876a-0736-4f36-bd1f-5b0f98f5f70d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:03.837717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 710/1410] 10.222.0.40 () {54 vars in 918 bytes} [Wed Aug 10 13:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:03.842850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631497-a31e-41a5-ba22-b5da45cc9408 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:03.843245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631497-a31e-41a5-ba22-b5da45cc9408 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:03.843401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631497-a31e-41a5-ba22-b5da45cc9408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:03.920364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0a631497-a31e-41a5-ba22-b5da45cc9408 None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:03.921506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0a631497-a31e-41a5-ba22-b5da45cc9408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:04.029968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 701/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 10 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:04.045437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef909dd-c682-40f3-a36d-fb519b59f27e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:04.045725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef909dd-c682-40f3-a36d-fb519b59f27e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:04.046003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef909dd-c682-40f3-a36d-fb519b59f27e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:04.047365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 711/1412] 10.222.0.40 () {54 vars in 918 bytes} [Wed Aug 10 13:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:04.052088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb3622-e5b9-4f8b-ac26-73bd9e70ec4c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:04.052372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb3622-e5b9-4f8b-ac26-73bd9e70ec4c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:04.052644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb3622-e5b9-4f8b-ac26-73bd9e70ec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:04.122980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:04.127095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-26fb3622-e5b9-4f8b-ac26-73bd9e70ec4c None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:04.128301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-26fb3622-e5b9-4f8b-ac26-73bd9e70ec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:04.241817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 702/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 10 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:06.706711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b6d61-606a-45a3-be1f-ec263893edb7 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:06.707252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b6d61-606a-45a3-be1f-ec263893edb7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:06.707252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b6d61-606a-45a3-be1f-ec263893edb7 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:06.708603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:06.713725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eafdfc-9d95-4e1b-b4f3-8137a2f22792 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:06.714151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eafdfc-9d95-4e1b-b4f3-8137a2f22792 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:06.714279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eafdfc-9d95-4e1b-b4f3-8137a2f22792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:06.779086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-01eafdfc-9d95-4e1b-b4f3-8137a2f22792 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:06.780531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-01eafdfc-9d95-4e1b-b4f3-8137a2f22792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:06.834510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:06.875602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:06.919864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b6d61-606a-45a3-be1f-ec263893edb7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:06.921345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-541b6d61-606a-45a3-be1f-ec263893edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:07.064615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b6d61-606a-45a3-be1f-ec263893edb7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:07.136276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:07.176012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:07.176551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:07.176832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:07.177137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:07.178732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:07.267640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:07.269266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f075c1-d201-4f65-8d60-55c841abb042 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:07.287165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 713/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:08.192238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c39ac-00d8-448b-8eba-7fd1582f1943 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:08.192822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c39ac-00d8-448b-8eba-7fd1582f1943 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:08.192822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c39ac-00d8-448b-8eba-7fd1582f1943 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:08.194102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:08.199163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae69f24-63fe-41b5-8298-50fe4f07f15f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:08.199603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae69f24-63fe-41b5-8298-50fe4f07f15f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:08.199716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae69f24-63fe-41b5-8298-50fe4f07f15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:08.274029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dae69f24-63fe-41b5-8298-50fe4f07f15f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:08.275247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dae69f24-63fe-41b5-8298-50fe4f07f15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:08.302843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:08.369875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:08.521552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c39ac-00d8-448b-8eba-7fd1582f1943 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:08.523083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-821c39ac-00d8-448b-8eba-7fd1582f1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:08.686885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c39ac-00d8-448b-8eba-7fd1582f1943 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:08.727012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:08.800121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:08.800653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:08.800932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:08.801235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:08.802834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:08.892142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:08.893777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2570b3-2e10-4efe-9702-bf0a89357bbe service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:08.912284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 705/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:09.818958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede9ce-54a0-4ed0-9ab9-2b0a0e74a346 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:09.819532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede9ce-54a0-4ed0-9ab9-2b0a0e74a346 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:09.819532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede9ce-54a0-4ed0-9ab9-2b0a0e74a346 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:09.820850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:09.826460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bbbc3-2a32-481f-aae8-5300b06b8b7b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:09.826850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bbbc3-2a32-481f-aae8-5300b06b8b7b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:09.827042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bbbc3-2a32-481f-aae8-5300b06b8b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:09.882178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:09.904851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-832bbbc3-2a32-481f-aae8-5300b06b8b7b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:09.906099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-832bbbc3-2a32-481f-aae8-5300b06b8b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:10.006797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:19:10.160120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede9ce-54a0-4ed0-9ab9-2b0a0e74a346 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:10.161708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7fede9ce-54a0-4ed0-9ab9-2b0a0e74a346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:10.303993 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede9ce-54a0-4ed0-9ab9-2b0a0e74a346 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:10.316422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:10.415714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:10.416309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:10.416583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:10.416849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:10.418779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:10.629494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:10.630998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177343d-5e7f-4276-8bf7-87e81a29e82d service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:10.649479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 716/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:11.515110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9791dc7-41e9-426c-b5d0-4809df2cfd5a None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:11.515643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9791dc7-41e9-426c-b5d0-4809df2cfd5a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:11.515643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9791dc7-41e9-426c-b5d0-4809df2cfd5a None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:11.517015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:11.521797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b50a-5721-48f3-bb36-f805a2669030 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:11.522229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b50a-5721-48f3-bb36-f805a2669030 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:11.522340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b50a-5721-48f3-bb36-f805a2669030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:11.542914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:11.597213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-23e8b50a-5721-48f3-bb36-f805a2669030 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:11.598348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-23e8b50a-5721-48f3-bb36-f805a2669030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:11.698054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:11.846047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9791dc7-41e9-426c-b5d0-4809df2cfd5a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:11.847642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d9791dc7-41e9-426c-b5d0-4809df2cfd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:11.956601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:11.992777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9791dc7-41e9-426c-b5d0-4809df2cfd5a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:12.107786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:12.108316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:12.108599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:12.108874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:12.110511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:12.185772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:12.198689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:12.201075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99650f0a-6ee9-4015-adb3-397ddb58ed61 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:12.218786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 708/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:19:13.093659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb402a30-97ad-4c3f-a743-6be1e4167724 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:13.094204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb402a30-97ad-4c3f-a743-6be1e4167724 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:13.094204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb402a30-97ad-4c3f-a743-6be1e4167724 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:13.095603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:13.100800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656809c-0bd7-44a7-868d-fb17ce0f971f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:13.101233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656809c-0bd7-44a7-868d-fb17ce0f971f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:13.101464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656809c-0bd7-44a7-868d-fb17ce0f971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:13.177628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a656809c-0bd7-44a7-868d-fb17ce0f971f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:13.178866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a656809c-0bd7-44a7-868d-fb17ce0f971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:13.291223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:13.442900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb402a30-97ad-4c3f-a743-6be1e4167724 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:13.444314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fb402a30-97ad-4c3f-a743-6be1e4167724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:13.531406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:13.592291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb402a30-97ad-4c3f-a743-6be1e4167724 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:13.707612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:13.708142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:13.708421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:13.708688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:13.710286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:13.757823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:13.799247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:13.800835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7191af3-bbcc-47bc-b7fe-e92310881854 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:13.818398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 719/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:14.720262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c8340-03e7-4120-8d89-cec07f6a3d3b None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:14.720825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c8340-03e7-4120-8d89-cec07f6a3d3b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:14.720825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c8340-03e7-4120-8d89-cec07f6a3d3b None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:14.722196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:14.727115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f62649-e65c-411e-9b17-6d4f4ca184ea None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:14.727567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f62649-e65c-411e-9b17-6d4f4ca184ea None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:14.727674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f62649-e65c-411e-9b17-6d4f4ca184ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:14.802559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-67f62649-e65c-411e-9b17-6d4f4ca184ea None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:14.803764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-67f62649-e65c-411e-9b17-6d4f4ca184ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:14.901412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:15.051247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c8340-03e7-4120-8d89-cec07f6a3d3b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:15.052793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-483c8340-03e7-4120-8d89-cec07f6a3d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:15.101373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:15.196650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c8340-03e7-4120-8d89-cec07f6a3d3b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:15.310061 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:15.310677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:15.310959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:15.311234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:15.312851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:15.319705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:15.404073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:15.405971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6646b86-2d82-4492-973c-3ab86e22d0d6 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:15.423779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 711/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:16.274265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b90e-6bc1-4149-945d-657ada54b0da None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:16.274797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b90e-6bc1-4149-945d-657ada54b0da None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:16.274797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b90e-6bc1-4149-945d-657ada54b0da None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:16.276169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:16.281709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdba4c-9556-47db-bddb-e1399ea8f89d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:16.282174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdba4c-9556-47db-bddb-e1399ea8f89d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:16.282282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdba4c-9556-47db-bddb-e1399ea8f89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:16.356442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6dfdba4c-9556-47db-bddb-e1399ea8f89d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:16.357768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6dfdba4c-9556-47db-bddb-e1399ea8f89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:16.423708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:16.452552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:16.603323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00b90e-6bc1-4149-945d-657ada54b0da None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:16.604881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e00b90e-6bc1-4149-945d-657ada54b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:16.622800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:16.751751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00b90e-6bc1-4149-945d-657ada54b0da None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:16.838559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:16.864002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:16.865192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:16.865192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:16.865192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:16.866732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:16.954537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:16.956044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4f419-9931-4067-8f37-011990fb8253 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:16.973762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 722/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:17.839654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd131-2ec7-4621-a83b-61d39fa60012 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:17.840332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd131-2ec7-4621-a83b-61d39fa60012 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:17.840332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd131-2ec7-4621-a83b-61d39fa60012 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:17.841877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:17.846832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e28f6-c280-4be4-9657-95dc759a74fb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:17.847261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e28f6-c280-4be4-9657-95dc759a74fb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:17.847389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e28f6-c280-4be4-9657-95dc759a74fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:17.922755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-129e28f6-c280-4be4-9657-95dc759a74fb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:17.923960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-129e28f6-c280-4be4-9657-95dc759a74fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:17.964327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:18.019083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:18.172198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd131-2ec7-4621-a83b-61d39fa60012 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:18.173665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bfcfd131-2ec7-4621-a83b-61d39fa60012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:18.318630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd131-2ec7-4621-a83b-61d39fa60012 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:18.372189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:18.432031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:18.432594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:18.432860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:18.433165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:18.434812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:18.522661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:18.524266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2136eb0-a74a-40b1-9094-9706df5ee96d service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:18.542790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 714/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:19:19.409900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17771-6f2e-446b-8463-64e1e2906227 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:19.410454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17771-6f2e-446b-8463-64e1e2906227 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:19.410454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17771-6f2e-446b-8463-64e1e2906227 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:19.411813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:19.416628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4abdf-2420-4506-a3bd-a60d3b721312 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:19.417143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4abdf-2420-4506-a3bd-a60d3b721312 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:19.417258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4abdf-2420-4506-a3bd-a60d3b721312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:19.492589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-65f4abdf-2420-4506-a3bd-a60d3b721312 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:19.493814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-65f4abdf-2420-4506-a3bd-a60d3b721312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:19.495799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:19.591140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:19.742825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc17771-6f2e-446b-8463-64e1e2906227 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:19.744356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9fc17771-6f2e-446b-8463-64e1e2906227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:19.885882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc17771-6f2e-446b-8463-64e1e2906227 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:19.912770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:19.998847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:19.999448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:19.999723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:19.999988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:20.001606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:20.096050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:20.097665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce12f-c473-4946-a8c9-aed9da828907 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:20.115447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 725/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:22.042088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bec3e3-8667-40c2-b7a0-5154f7df6ab7 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:22.042612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bec3e3-8667-40c2-b7a0-5154f7df6ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:22.042736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bec3e3-8667-40c2-b7a0-5154f7df6ab7 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:22.044344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:22.049693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809783f8-7f54-467a-b0e2-3c48ddc10a70 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:22.050124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809783f8-7f54-467a-b0e2-3c48ddc10a70 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:22.050221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809783f8-7f54-467a-b0e2-3c48ddc10a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:22.092861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:22.118056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-809783f8-7f54-467a-b0e2-3c48ddc10a70 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:22.119256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-809783f8-7f54-467a-b0e2-3c48ddc10a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:22.214226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:22.253435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bec3e3-8667-40c2-b7a0-5154f7df6ab7 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:22.255020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-82bec3e3-8667-40c2-b7a0-5154f7df6ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:22.357729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bec3e3-8667-40c2-b7a0-5154f7df6ab7 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:22.389478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:22.430270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:22.430881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:22.431158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:22.431428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:22.433040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:22.541196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:22.542637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c268670-1d89-48b2-a789-fa4e2c766c5a service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:22.560756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 717/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:19:24.722503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33919-1b3b-4c5d-938a-1b1573202114 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:24.723071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33919-1b3b-4c5d-938a-1b1573202114 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:24.723071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33919-1b3b-4c5d-938a-1b1573202114 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:24.724394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:24.729696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53742074-3caa-4500-89c4-24ac44d42c80 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:24.730143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53742074-3caa-4500-89c4-24ac44d42c80 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:24.730267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53742074-3caa-4500-89c4-24ac44d42c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:24.795669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-53742074-3caa-4500-89c4-24ac44d42c80 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:24.796893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-53742074-3caa-4500-89c4-24ac44d42c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:24.810036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:24.893605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:24.932786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b33919-1b3b-4c5d-938a-1b1573202114 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:24.935089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f2b33919-1b3b-4c5d-938a-1b1573202114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:25.037779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b33919-1b3b-4c5d-938a-1b1573202114 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:25.049857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:25.110446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:25.111052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:25.111333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:25.111603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:25.113233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:25.220562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:25.222023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49417-367b-425c-a7ee-6d276c51b9da service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:25.240011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 728/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:27.575088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c7f9a-d2bb-49ba-8cd9-f1039db58e01 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:27.575616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c7f9a-d2bb-49ba-8cd9-f1039db58e01 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:27.575616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c7f9a-d2bb-49ba-8cd9-f1039db58e01 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:27.577028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:27.582314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b19fff-e58b-481e-b91c-faf2b25465a2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:27.582738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b19fff-e58b-481e-b91c-faf2b25465a2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:27.582834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b19fff-e58b-481e-b91c-faf2b25465a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:27.630347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:27.648160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f5b19fff-e58b-481e-b91c-faf2b25465a2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:27.649909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f5b19fff-e58b-481e-b91c-faf2b25465a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:27.748048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:27.788857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c7f9a-d2bb-49ba-8cd9-f1039db58e01 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:27.790420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-225c7f9a-d2bb-49ba-8cd9-f1039db58e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:27.893070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c7f9a-d2bb-49ba-8cd9-f1039db58e01 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:27.931679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:27.967787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:27.968531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:27.968808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:27.969119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:27.970689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:28.080185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:28.081650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d9b93-d590-42bb-b38a-3406050b1d77 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:28.100954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 720/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:30.275084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca9fac-a1df-480e-b52c-21a92b89bc03 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:30.275648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca9fac-a1df-480e-b52c-21a92b89bc03 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:30.275648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca9fac-a1df-480e-b52c-21a92b89bc03 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:30.277027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:30.283561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6e2ac-8500-44cf-ab9c-22ecf903cbb7 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:30.283944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6e2ac-8500-44cf-ab9c-22ecf903cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:30.284138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6e2ac-8500-44cf-ab9c-22ecf903cbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:30.350232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-22b6e2ac-8500-44cf-ab9c-22ecf903cbb7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:30.351412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-22b6e2ac-8500-44cf-ab9c-22ecf903cbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:30.372059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:30.447208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:30.487524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca9fac-a1df-480e-b52c-21a92b89bc03 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:30.489086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f0ca9fac-a1df-480e-b52c-21a92b89bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:30.590788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca9fac-a1df-480e-b52c-21a92b89bc03 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:30.610801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:30.666311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:30.666920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:30.667201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:30.667498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:30.669983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:30.776978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:30.778969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1d6ca-9ff1-4d5c-84e1-4a996e08a0e2 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:30.797183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 731/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:32.845881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762f43c-6912-493e-8bcc-a4782abec6e4 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:32.846445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762f43c-6912-493e-8bcc-a4782abec6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:32.846445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762f43c-6912-493e-8bcc-a4782abec6e4 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:32.847795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:32.853079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9928-d6e4-4e58-8d8a-339aa37c7b25 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:32.853519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9928-d6e4-4e58-8d8a-339aa37c7b25 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:32.853626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9928-d6e4-4e58-8d8a-339aa37c7b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:32.918004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:32.920612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-baac9928-d6e4-4e58-8d8a-339aa37c7b25 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:32.921857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-baac9928-d6e4-4e58-8d8a-339aa37c7b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:33.017494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:33.055928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762f43c-6912-493e-8bcc-a4782abec6e4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:33.057474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2762f43c-6912-493e-8bcc-a4782abec6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:33.163602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762f43c-6912-493e-8bcc-a4782abec6e4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:33.211171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:33.235015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:33.235623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:33.235905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:33.236180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:33.237886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:33.345824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:33.347261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe50e1-7966-4097-a6c0-d123d9bbd4b7 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:33.366014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 723/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:35.408434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833686-08d7-446e-9e2b-e461088a6bfd None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:35.408975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833686-08d7-446e-9e2b-e461088a6bfd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:35.408975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833686-08d7-446e-9e2b-e461088a6bfd None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:35.410330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:35.415414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2438c-1e94-43e7-ab4a-e4f736d4a832 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:35.415811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2438c-1e94-43e7-ab4a-e4f736d4a832 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:35.415964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2438c-1e94-43e7-ab4a-e4f736d4a832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:35.482612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f7d2438c-1e94-43e7-ab4a-e4f736d4a832 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:35.483812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f7d2438c-1e94-43e7-ab4a-e4f736d4a832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:35.513348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:35.581497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:35.621328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833686-08d7-446e-9e2b-e461088a6bfd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:35.624910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-60833686-08d7-446e-9e2b-e461088a6bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:35.727027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833686-08d7-446e-9e2b-e461088a6bfd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:35.755504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:35.800241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:35.800856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:35.801172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:35.801450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:35.803040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:35.910918 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:35.912483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2112a-67ce-4c3a-8936-644c79ff5c4e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:35.932974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 734/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:37.987236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06908837-309f-4863-8fbe-0a054ff22753 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:37.987750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06908837-309f-4863-8fbe-0a054ff22753 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:37.987750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06908837-309f-4863-8fbe-0a054ff22753 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:37.989064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:37.994171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385153ce-92b7-458b-a11f-fde5d9c2fab6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:37.994593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385153ce-92b7-458b-a11f-fde5d9c2fab6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:37.994698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385153ce-92b7-458b-a11f-fde5d9c2fab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:38.061942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-385153ce-92b7-458b-a11f-fde5d9c2fab6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:38.063154 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-385153ce-92b7-458b-a11f-fde5d9c2fab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:38.071191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:38.160345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:38.197331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06908837-309f-4863-8fbe-0a054ff22753 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:38.199498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-06908837-309f-4863-8fbe-0a054ff22753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:38.302463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06908837-309f-4863-8fbe-0a054ff22753 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:38.355997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:38.374290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:38.374898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:38.375175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:38.375444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:38.377049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:38.491604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:38.493098 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c43373-fe28-4665-a257-6db55dca9d50 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:38.512252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 726/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:40.575671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe3bdb-dae1-4462-b3c5-27020ae4b7a5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:40.576213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe3bdb-dae1-4462-b3c5-27020ae4b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:40.576213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe3bdb-dae1-4462-b3c5-27020ae4b7a5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:40.577621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:19:40.582916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843fcd6-b798-48cc-8659-ca9c405cc645 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:40.583379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843fcd6-b798-48cc-8659-ca9c405cc645 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:40.583493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843fcd6-b798-48cc-8659-ca9c405cc645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:40.650077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1843fcd6-b798-48cc-8659-ca9c405cc645 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:40.651311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1843fcd6-b798-48cc-8659-ca9c405cc645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:40.813832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:40.872280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:40.910777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe3bdb-dae1-4462-b3c5-27020ae4b7a5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:40.912330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-62fe3bdb-dae1-4462-b3c5-27020ae4b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:41.016134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe3bdb-dae1-4462-b3c5-27020ae4b7a5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:41.053307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:41.089127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:41.089812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:41.090083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:41.090348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:41.091943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:41.198340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:41.200331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ceea-352e-4418-a61f-b2e095d35990 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:41.219157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 737/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:43.263679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1dd4e-2add-48fc-92c2-c679d4f07fc0 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:43.264217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1dd4e-2add-48fc-92c2-c679d4f07fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:43.264217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1dd4e-2add-48fc-92c2-c679d4f07fc0 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:43.265605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:19:43.270715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc469d-a2f2-472a-8c04-dfdd36de5db5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:43.271135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc469d-a2f2-472a-8c04-dfdd36de5db5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:43.271268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc469d-a2f2-472a-8c04-dfdd36de5db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:43.336302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a6dc469d-a2f2-472a-8c04-dfdd36de5db5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:43.337581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a6dc469d-a2f2-472a-8c04-dfdd36de5db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:43.356596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:43.434688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:43.471582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1dd4e-2add-48fc-92c2-c679d4f07fc0 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:43.473147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-63c1dd4e-2add-48fc-92c2-c679d4f07fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:43.575195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1dd4e-2add-48fc-92c2-c679d4f07fc0 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:43.639714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:43.646321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:43.650458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:43.650458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:43.650458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:43.650458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:43.761663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:43.763002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a618d44-002d-44aa-8a40-380be8f63b64 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:43.782230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 729/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:45.790937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5aeb1-2637-480a-91ef-bf7e3c605650 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:45.790937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5aeb1-2637-480a-91ef-bf7e3c605650 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:45.790937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5aeb1-2637-480a-91ef-bf7e3c605650 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:45.792915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:19:45.798199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f825d2-5088-42ee-8574-41636646fa9e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:45.798643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f825d2-5088-42ee-8574-41636646fa9e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:45.798801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f825d2-5088-42ee-8574-41636646fa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:45.864410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d9f825d2-5088-42ee-8574-41636646fa9e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:45.865647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d9f825d2-5088-42ee-8574-41636646fa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:45.913505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:45.961068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:19:45.998478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5aeb1-2637-480a-91ef-bf7e3c605650 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:46.000003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9ce5aeb1-2637-480a-91ef-bf7e3c605650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:46.101444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5aeb1-2637-480a-91ef-bf7e3c605650 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:46.155585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:46.181376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:46.181911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:46.182410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:46.183666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:46.185746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:46.291342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:46.293456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74050c87-879c-44b6-b13e-7d2ab1b5a160 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:46.312028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 740/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:55.959314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=22 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:56.309665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=66 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:06.107940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a42546-5540-4d45-8aaf-a9143cc602b5 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:06.108593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a42546-5540-4d45-8aaf-a9143cc602b5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:06.108593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a42546-5540-4d45-8aaf-a9143cc602b5 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:06.109878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 731/1471] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:06.115120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c981bf-92d9-4318-800f-aa139ad00881 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:06.115529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c981bf-92d9-4318-800f-aa139ad00881 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:06.115665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c981bf-92d9-4318-800f-aa139ad00881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:06.179336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-93c981bf-92d9-4318-800f-aa139ad00881 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:06.180525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-93c981bf-92d9-4318-800f-aa139ad00881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:06.276348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 741/1472] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:06.362116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1304588-1ae2-4673-83f4-2f249bcaedbf None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:06.362344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1304588-1ae2-4673-83f4-2f249bcaedbf None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:06.362617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1304588-1ae2-4673-83f4-2f249bcaedbf None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:06.363942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 732/1473] 10.222.0.40 () {54 vars in 967 bytes} [Wed Aug 10 13:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:06.368987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0df1-bf09-4e01-ab12-fb4641fd25b0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:06.369297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0df1-bf09-4e01-ab12-fb4641fd25b0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:06.369567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0df1-bf09-4e01-ab12-fb4641fd25b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:06.447605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-926b0df1-bf09-4e01-ab12-fb4641fd25b0 None None] MFA Rules not processed for user `8279c74cfda94068b5b392123282e556`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:06.448819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-926b0df1-bf09-4e01-ab12-fb4641fd25b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:06.469112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:06.524463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 742/1474] 10.222.0.40 () {58 vars in 1064 bytes} [Wed Aug 10 13:20:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:06.532220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1304588-1ae2-4673-83f4-2f249bcaedbf None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:06.534015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d1304588-1ae2-4673-83f4-2f249bcaedbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:06.641467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1304588-1ae2-4673-83f4-2f249bcaedbf None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:06.715997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:06.716606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:06.716883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:06.717184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:06.718775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:06.772046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:06.826286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:06.827771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1a008-b515-4f15-97d7-b25e6519cb8e service glance] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:06.845961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 733/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:08.978271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac78958-fbe2-460e-ae7a-355dc42de390 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:08.978782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac78958-fbe2-460e-ae7a-355dc42de390 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:08.978782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac78958-fbe2-460e-ae7a-355dc42de390 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:08.980131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 743/1476] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:08.985805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864cdd6-1ec4-4fcc-b2da-49f54b9a608f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:08.986246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864cdd6-1ec4-4fcc-b2da-49f54b9a608f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:08.986350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864cdd6-1ec4-4fcc-b2da-49f54b9a608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:09.050949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d864cdd6-1ec4-4fcc-b2da-49f54b9a608f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:09.052158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d864cdd6-1ec4-4fcc-b2da-49f54b9a608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:09.148643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 734/1477] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:09.172671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac78958-fbe2-460e-ae7a-355dc42de390 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:09.174251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5ac78958-fbe2-460e-ae7a-355dc42de390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:09.296157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac78958-fbe2-460e-ae7a-355dc42de390 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:09.344233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:09.389222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:09.389835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:09.390114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:09.390386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:09.391029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:09.392762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:20:09.392762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:20:09.392762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:20:09.393938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3262835-3a2f-4bd4-ab39-10c782455e71 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:09.412584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 744/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 10 13:20:09 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:20:11.429019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a98206-6a06-4c90-bd91-0c9dddab7233 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:11.429544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a98206-6a06-4c90-bd91-0c9dddab7233 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:11.429640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a98206-6a06-4c90-bd91-0c9dddab7233 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:11.431060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 735/1479] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:11.437574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddcd8c-d15d-474d-a49e-1b431d78bb51 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:11.437992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddcd8c-d15d-474d-a49e-1b431d78bb51 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:11.438134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddcd8c-d15d-474d-a49e-1b431d78bb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:11.503664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-20ddcd8c-d15d-474d-a49e-1b431d78bb51 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:11.504881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-20ddcd8c-d15d-474d-a49e-1b431d78bb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:11.602110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 745/1480] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:11.645421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a98206-6a06-4c90-bd91-0c9dddab7233 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:11.647047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f2a98206-6a06-4c90-bd91-0c9dddab7233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:11.660171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:11.796649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a98206-6a06-4c90-bd91-0c9dddab7233 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:11.876949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:11.911173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:11.911698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:11.911973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:11.912240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:11.913840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:12.002036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:12.003624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c369d5-b3f3-44d0-a032-95f4c13b8633 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:12.021718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 736/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:14.211705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ae11c-4a25-4aab-8894-a1a034d67217 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:14.212245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ae11c-4a25-4aab-8894-a1a034d67217 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:14.212245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ae11c-4a25-4aab-8894-a1a034d67217 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:14.213674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 746/1482] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:20:14.219342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534666c-c5ea-4c2e-b35c-4c6333af1efc None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:14.219811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534666c-c5ea-4c2e-b35c-4c6333af1efc None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:14.219918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534666c-c5ea-4c2e-b35c-4c6333af1efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:14.289016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2534666c-c5ea-4c2e-b35c-4c6333af1efc None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:14.290214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2534666c-c5ea-4c2e-b35c-4c6333af1efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:14.335743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:14.385408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 737/1483] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:14.423108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ae11c-4a25-4aab-8894-a1a034d67217 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:14.424634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bb6ae11c-4a25-4aab-8894-a1a034d67217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:14.448035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:14.529928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ae11c-4a25-4aab-8894-a1a034d67217 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:14.602080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:14.602688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:14.602968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:14.603240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:14.605365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:14.680536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:14.715839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:14.717357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0ad02-c144-4504-a98e-03a598f116aa service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:14.735550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 747/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:16.829877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279ee5a-6743-4ced-b006-f3786248ceb2 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:16.830474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279ee5a-6743-4ced-b006-f3786248ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:16.830474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279ee5a-6743-4ced-b006-f3786248ceb2 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:16.831779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 738/1485] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:16.837143 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1f65c-6bc4-4a0b-967e-1dafb393a8bd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:16.837553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1f65c-6bc4-4a0b-967e-1dafb393a8bd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:16.837690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1f65c-6bc4-4a0b-967e-1dafb393a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:16.904549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6da1f65c-6bc4-4a0b-967e-1dafb393a8bd None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:16.905984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6da1f65c-6bc4-4a0b-967e-1dafb393a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:17.002493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 748/1486] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:17.040770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279ee5a-6743-4ced-b006-f3786248ceb2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:17.042843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d279ee5a-6743-4ced-b006-f3786248ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:17.145838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279ee5a-6743-4ced-b006-f3786248ceb2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:17.216793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:17.218647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:17.219249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:17.219527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:17.219798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:17.221407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:17.327667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:17.329112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822872-e887-4259-a29e-7feb56a8b321 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:17.347104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 739/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:19.525348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f3cb8-a1a1-42bf-8819-b4ae86af5a58 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:19.525921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f3cb8-a1a1-42bf-8819-b4ae86af5a58 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:19.525921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f3cb8-a1a1-42bf-8819-b4ae86af5a58 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:19.527868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 749/1488] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:19.533368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd212c-52ca-416a-b61b-90777a5c5288 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:19.533826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd212c-52ca-416a-b61b-90777a5c5288 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:19.533943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd212c-52ca-416a-b61b-90777a5c5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:19.598759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-3acd212c-52ca-416a-b61b-90777a5c5288 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:19.599969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3acd212c-52ca-416a-b61b-90777a5c5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:19.653371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:19.696969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 740/1489] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:20:19.737742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f3cb8-a1a1-42bf-8819-b4ae86af5a58 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:19.739276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-659f3cb8-a1a1-42bf-8819-b4ae86af5a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:19.775406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:19.844241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f3cb8-a1a1-42bf-8819-b4ae86af5a58 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:19.916912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:19.918488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:19.918773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:19.919042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:19.920658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:20.000158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:20.028278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:20.030040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f513f27-e182-49f2-bb25-d28efa5df64b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:20.048280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 750/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:22.473768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b999aa-2454-48c5-a624-019d60906ce6 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:22.474333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b999aa-2454-48c5-a624-019d60906ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:22.474333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b999aa-2454-48c5-a624-019d60906ce6 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:22.475742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 741/1491] 10.222.0.40 () {54 vars in 927 bytes} [Wed Aug 10 13:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:20:22.481754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2661a6-178f-4d6e-b69f-e25e3205a872 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:22.482163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2661a6-178f-4d6e-b69f-e25e3205a872 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:22.482294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2661a6-178f-4d6e-b69f-e25e3205a872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:22.558118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bf2661a6-178f-4d6e-b69f-e25e3205a872 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:20:22.559523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bf2661a6-178f-4d6e-b69f-e25e3205a872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:22.668581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 751/1492] 10.222.0.40 () {58 vars in 1024 bytes} [Wed Aug 10 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:20:22.707779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b999aa-2454-48c5-a624-019d60906ce6 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:22.709337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f4b999aa-2454-48c5-a624-019d60906ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:22.812489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b999aa-2454-48c5-a624-019d60906ce6 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:22.885411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:22.887065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:22.887351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:22.887622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:22.889233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:22.898079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:23.005006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:23.006457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9974ecd-3183-4991-9689-c65344bfd224 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:23.024892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 742/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:20:32.665489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=82 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:33.022202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=50 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:42.609206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:42.609811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:42.609811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:42.688905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:21:42.690175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:42.791818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 752/1494] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:21:42.800106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:42.800587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:42.800696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:42.877886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:21:42.879003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:42.975173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 743/1495] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:21:42.980691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:42.981944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:42.993865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f8e01-924b-464f-91c4-c4fbaa1c5cf9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:43.055278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:43.089170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:43.090450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:43.090450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:43.090450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:43.091010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:21:43.091304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:43.092291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:43.092291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:21:43.092291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:21:43.093290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:43.105267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 753/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:21:43.113828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:43.123685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31bd33-6294-4d4e-8b26-e17eeeb41f46 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:43.183941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:43.215957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:43.216560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:43.216835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:43.217131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:43.217742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:43.219439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:43.219439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:21:43.219439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:21:43.220617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:43.332788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 744/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:21:43 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:21:43.337390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:43.338908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:43.372379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eea1d-cdca-4f31-adf7-35a0a2ee15a9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:43.464182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:43.464725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:43.465021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:43.465307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:43.465921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:43.466921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:43.466921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:21:43.466921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:21:43.468129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:43.606526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 754/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:21:43 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:21:43.610837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:43.612337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:43.674538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d832f-d453-4762-85c7-47b7f64c6d4b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:43.756558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:43.766953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:43.767561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:43.767838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:43.768107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:43.768747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:43.769786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:43.769786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:21:43.769786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:21:43.770677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:43.783685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 745/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:21:43.787679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:43.789219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:43.801763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65832608-c47c-4fbc-a736-320bd8a9a410 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:43.827782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:43.896522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:43.897161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:43.897445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:43.897725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:43.898365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:43.899588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:43.899588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:21:43.899588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:21:43.900474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864732a9-cb13-487b-b6e0-6662d1b1462d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:43.913352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 755/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:21:43.918075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:43.919622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:43.953348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7761bf-3cf1-4f56-8d25-dfb4417639e6 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:44.045947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:44.046613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:44.046894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:44.047166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] PATH_INFO: `/v3/projects/53610fd33d0c4899b99a848123ef6a7c/users/6e0754522b2b4b6bb73d7856e987770c/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:44.068664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=53610fd33d0c4899b99a848123ef6a7c, user_id=6e0754522b2b4b6bb73d7856e987770c, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:44.069725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:44.069725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:21:44.069725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:21:44.071855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7991937-a9eb-49b1-a1ce-f49ae58860ea admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:44.152862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 746/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:21:43 2022] PUT /identity/v3/projects/53610fd33d0c4899b99a848123ef6a7c/users/6e0754522b2b4b6bb73d7856e987770c/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:21:44.165690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d628d73-820b-4346-b07e-a7d8084b0e3f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:44.166292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d628d73-820b-4346-b07e-a7d8084b0e3f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:44.166577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d628d73-820b-4346-b07e-a7d8084b0e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:44.223138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5d628d73-820b-4346-b07e-a7d8084b0e3f None None] MFA Rules not processed for user `6e0754522b2b4b6bb73d7856e987770c`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:21:44.224331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5d628d73-820b-4346-b07e-a7d8084b0e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:44.298972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 756/1502] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:21:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:21:44.312519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3636ad-85e4-499c-aa27-2a7fc9c7e590 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:44.312805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3636ad-85e4-499c-aa27-2a7fc9c7e590 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:44.313115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3636ad-85e4-499c-aa27-2a7fc9c7e590 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:44.314451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 747/1503] 10.222.0.40 () {54 vars in 964 bytes} [Wed Aug 10 13:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:21:44.319368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3254f-0069-45c1-949a-de70a0f25159 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:44.319658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3254f-0069-45c1-949a-de70a0f25159 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:44.319933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3254f-0069-45c1-949a-de70a0f25159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:44.350784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:44.394158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-21b3254f-0069-45c1-949a-de70a0f25159 None None] MFA Rules not processed for user `64ff5fa2e326485bb2f915fa4f8870fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:21:44.395363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-21b3254f-0069-45c1-949a-de70a0f25159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:44.470658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 757/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Aug 10 13:21:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:21:44.479012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3636ad-85e4-499c-aa27-2a7fc9c7e590 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:44.480509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cc3636ad-85e4-499c-aa27-2a7fc9c7e590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:44.531796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:44.586235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3636ad-85e4-499c-aa27-2a7fc9c7e590 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:44.658173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:44.658798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:44.659128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:44.659402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:44.661069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:44.763791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:44.765241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:44.768622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:44.783854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 748/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:21:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:21:44.832206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edccfb-b10c-436f-b6f6-b3e5ccb2d26e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:44.832678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edccfb-b10c-436f-b6f6-b3e5ccb2d26e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:44.832882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edccfb-b10c-436f-b6f6-b3e5ccb2d26e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:44.834235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 758/1506] 10.222.0.40 () {52 vars in 830 bytes} [Wed Aug 10 13:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:21:44.838174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:44.839350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:44.937718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91bfd-9e14-4a90-acac-19577a035fc9 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:45.029331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:45.029966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:45.030247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:45.030776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] PATH_INFO: `/v3/users/6e0754522b2b4b6bb73d7856e987770c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:45.043463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0754522b2b4b6bb73d7856e987770c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:45.044488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:45.044488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:21:45.044488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:21:45.045743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:45.055090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:45.276617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0754522b2b4b6bb73d7856e987770c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:45.353779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0754522b2b4b6bb73d7856e987770c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:45.368707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0754522b2b4b6bb73d7856e987770c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:45.369189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0754522b2b4b6bb73d7856e987770c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:45.392418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0754522b2b4b6bb73d7856e987770c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:45.398123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 749/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:21:44 2022] DELETE /identity/v3/users/6e0754522b2b4b6bb73d7856e987770c => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:21:45.402907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:45.403195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:45.403477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:45.460833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:21:45.462670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:45.537087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:45.549142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 759/1508] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:21:45.558951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:45.560174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:45.665558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c6c3d-d31c-4aab-8cf7-18e51eb64a07 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:45.720264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:45.739080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:45.739693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:45.739975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:45.740246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:45.741883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:45.847119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:45.848489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3882e-8549-47de-86f3-a4762b380513 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:45.867463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 750/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:21:46.348372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:46.349622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:46.361803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3bdb2-621b-4f14-8602-c8d30a6ab1a1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:46.458578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:46.459210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:46.459487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:46.459761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] PATH_INFO: `/v3/projects/53610fd33d0c4899b99a848123ef6a7c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:46.471618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53610fd33d0c4899b99a848123ef6a7c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:46.472637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:21:46.472637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:21:46.472637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:21:46.473882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:46.584708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53610fd33d0c4899b99a848123ef6a7c', 'request_id': 'req-7dfc4df7-f5ca-4326-ab23-794755dd4393'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:46.591348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53610fd33d0c4899b99a848123ef6a7c', 'request_id': 'req-7dfc4df7-f5ca-4326-ab23-794755dd4393'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:46.604972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53610fd33d0c4899b99a848123ef6a7c', 'request_id': 'req-7dfc4df7-f5ca-4326-ab23-794755dd4393'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:46.668881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] The token cache is being invalidate because project 53610fd33d0c4899b99a848123ef6a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:21:46.669215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:46.669648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7dfc4df7-f5ca-4326-ab23-794755dd4393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:21:46.671066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 760/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:21:46 2022] DELETE /identity/v3/projects/53610fd33d0c4899b99a848123ef6a7c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:21:56.182686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=62 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:56.666777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:17.039165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bf981-466a-457d-82d4-4eeff3435bd5 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:17.039777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bf981-466a-457d-82d4-4eeff3435bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:17.039946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bf981-466a-457d-82d4-4eeff3435bd5 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:17.041729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:17.046853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438f361-4e5d-4872-8743-a0d4a91b36b7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:17.047353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438f361-4e5d-4872-8743-a0d4a91b36b7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:17.047458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438f361-4e5d-4872-8743-a0d4a91b36b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:17.127518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2438f361-4e5d-4872-8743-a0d4a91b36b7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:17.128730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2438f361-4e5d-4872-8743-a0d4a91b36b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:17.230977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:23:17.384237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bf981-466a-457d-82d4-4eeff3435bd5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:17.397257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d94bf981-466a-457d-82d4-4eeff3435bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:17.504435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bf981-466a-457d-82d4-4eeff3435bd5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:17.510914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b0a00-8b76-498b-bdf3-f732fb16174f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:17.511514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b0a00-8b76-498b-bdf3-f732fb16174f service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:17.511794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b0a00-8b76-498b-bdf3-f732fb16174f service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:17.512067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b0a00-8b76-498b-bdf3-f732fb16174f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:17.519959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b0a00-8b76-498b-bdf3-f732fb16174f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:17.521562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b0a00-8b76-498b-bdf3-f732fb16174f service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:17.530996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 752/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:18.369255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e746-2888-4009-9f62-5d4482ea3835 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:18.369815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e746-2888-4009-9f62-5d4482ea3835 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:18.369815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e746-2888-4009-9f62-5d4482ea3835 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:18.371158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:18.375895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce8eb8-314b-4c53-ac39-39b7b0386fa7 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:18.376342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce8eb8-314b-4c53-ac39-39b7b0386fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:18.376472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce8eb8-314b-4c53-ac39-39b7b0386fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:18.395696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8fce8eb8-314b-4c53-ac39-39b7b0386fa7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:18.396812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8fce8eb8-314b-4c53-ac39-39b7b0386fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:18.404635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:18.567114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e746-2888-4009-9f62-5d4482ea3835 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:18.581902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e746-2888-4009-9f62-5d4482ea3835 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:18.588127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c246e9-bcc2-4ae7-b4fd-d7e4c7d7603b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:18.588710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c246e9-bcc2-4ae7-b4fd-d7e4c7d7603b service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:18.589014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c246e9-bcc2-4ae7-b4fd-d7e4c7d7603b service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:18.589283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c246e9-bcc2-4ae7-b4fd-d7e4c7d7603b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:18.597577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c246e9-bcc2-4ae7-b4fd-d7e4c7d7603b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:18.599196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c246e9-bcc2-4ae7-b4fd-d7e4c7d7603b service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:18.608554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 763/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:19.388299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4cd64-fc62-4b45-b8bc-164551558164 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:19.388863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4cd64-fc62-4b45-b8bc-164551558164 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:19.388863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4cd64-fc62-4b45-b8bc-164551558164 None None] PATH_INFO: `/v3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:19.390161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 754/1517] 10.222.0.13 () {54 vars in 925 bytes} [Wed Aug 10 13:23:19 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:23:19.415278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66886f6-5c89-4f69-aa94-f1383a8a653e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:19.415738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66886f6-5c89-4f69-aa94-f1383a8a653e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:19.415863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66886f6-5c89-4f69-aa94-f1383a8a653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:19.473895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d66886f6-5c89-4f69-aa94-f1383a8a653e None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:19.475117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d66886f6-5c89-4f69-aa94-f1383a8a653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:19.492360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:19.545537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 764/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 10 13:23:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:23.588868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787ef2-655d-466b-9211-45d84c4d8390 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:23.589435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787ef2-655d-466b-9211-45d84c4d8390 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:23.589435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787ef2-655d-466b-9211-45d84c4d8390 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:23.590693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 755/1519] 10.222.0.40 () {54 vars in 961 bytes} [Wed Aug 10 13:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:23.595773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c86802-9042-48c5-8bcf-03e859e9399c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:23.596228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c86802-9042-48c5-8bcf-03e859e9399c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:23.596336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c86802-9042-48c5-8bcf-03e859e9399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:23.616861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-39c86802-9042-48c5-8bcf-03e859e9399c None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:23.618185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-39c86802-9042-48c5-8bcf-03e859e9399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:23.626523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 765/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Aug 10 13:23:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:23.634365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787ef2-655d-466b-9211-45d84c4d8390 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:23.647860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787ef2-655d-466b-9211-45d84c4d8390 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:23.655477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d17e3-bd86-47aa-9854-cefebb1876c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:23.656602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d17e3-bd86-47aa-9854-cefebb1876c1 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:23.657043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d17e3-bd86-47aa-9854-cefebb1876c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:23.657460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d17e3-bd86-47aa-9854-cefebb1876c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:23.666906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d17e3-bd86-47aa-9854-cefebb1876c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:23.668483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d17e3-bd86-47aa-9854-cefebb1876c1 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:23.678187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 756/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:29.845492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d5e-68a0-4e0b-97f2-e0a10e21f371 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:29.846063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d5e-68a0-4e0b-97f2-e0a10e21f371 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:29.846063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d5e-68a0-4e0b-97f2-e0a10e21f371 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:29.847418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:29.853039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70527580-ea83-477a-b253-16742dbf0fbb None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:29.853546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70527580-ea83-477a-b253-16742dbf0fbb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:29.853651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70527580-ea83-477a-b253-16742dbf0fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:29.873731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-70527580-ea83-477a-b253-16742dbf0fbb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:29.874931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-70527580-ea83-477a-b253-16742dbf0fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:29.882736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:30.038232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d5e-68a0-4e0b-97f2-e0a10e21f371 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:30.053604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d5e-68a0-4e0b-97f2-e0a10e21f371 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:30.059178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f98fa9-cd84-470d-a1f1-b465ec58d230 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:30.059768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f98fa9-cd84-470d-a1f1-b465ec58d230 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:30.060048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f98fa9-cd84-470d-a1f1-b465ec58d230 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:30.060318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f98fa9-cd84-470d-a1f1-b465ec58d230 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:30.068371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f98fa9-cd84-470d-a1f1-b465ec58d230 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:30.073637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f98fa9-cd84-470d-a1f1-b465ec58d230 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:30.079535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 767/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:30.909410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fafc2-47eb-4b35-99ed-13896d73cfe3 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:30.909950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fafc2-47eb-4b35-99ed-13896d73cfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:30.909950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fafc2-47eb-4b35-99ed-13896d73cfe3 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:30.911233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:30.916422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c007d3b-2d20-4bfa-9a48-4aba73fd2638 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:30.916840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c007d3b-2d20-4bfa-9a48-4aba73fd2638 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:30.916968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c007d3b-2d20-4bfa-9a48-4aba73fd2638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:30.936608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4c007d3b-2d20-4bfa-9a48-4aba73fd2638 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:30.937833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4c007d3b-2d20-4bfa-9a48-4aba73fd2638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:30.946260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:23:31.099065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fafc2-47eb-4b35-99ed-13896d73cfe3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:31.113843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fafc2-47eb-4b35-99ed-13896d73cfe3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:31.120079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa971bf-39c7-44e6-a1d2-67d9f588966d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:31.120668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa971bf-39c7-44e6-a1d2-67d9f588966d service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:31.120948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa971bf-39c7-44e6-a1d2-67d9f588966d service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:31.121248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa971bf-39c7-44e6-a1d2-67d9f588966d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:31.129106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa971bf-39c7-44e6-a1d2-67d9f588966d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:31.130676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa971bf-39c7-44e6-a1d2-67d9f588966d service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:31.141081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 759/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:35.000367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae4ef7-6a65-4ff1-9183-88072cb5bd32 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:35.000950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae4ef7-6a65-4ff1-9183-88072cb5bd32 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:35.000950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae4ef7-6a65-4ff1-9183-88072cb5bd32 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:35.002319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:35.007323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325484e-4fe3-472b-8c8e-51e3c32c1156 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:35.008236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325484e-4fe3-472b-8c8e-51e3c32c1156 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:35.008459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325484e-4fe3-472b-8c8e-51e3c32c1156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:35.028519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7325484e-4fe3-472b-8c8e-51e3c32c1156 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:35.029780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7325484e-4fe3-472b-8c8e-51e3c32c1156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:35.037967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:35.111150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae4ef7-6a65-4ff1-9183-88072cb5bd32 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:35.113774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9eae4ef7-6a65-4ff1-9183-88072cb5bd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:35.194395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae4ef7-6a65-4ff1-9183-88072cb5bd32 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:35.199067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503723c-931e-4328-a51a-62c1278b4f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:35.199671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503723c-931e-4328-a51a-62c1278b4f75 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:35.199945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503723c-931e-4328-a51a-62c1278b4f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:35.200210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503723c-931e-4328-a51a-62c1278b4f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:35.208827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503723c-931e-4328-a51a-62c1278b4f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:35.210285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503723c-931e-4328-a51a-62c1278b4f75 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:35.220087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 770/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:37.270826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd30e2-33f6-4dc8-87c0-7d0bceb1544a None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:37.271397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd30e2-33f6-4dc8-87c0-7d0bceb1544a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:37.271397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd30e2-33f6-4dc8-87c0-7d0bceb1544a None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:37.272766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:37.278317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476ead-d1c9-48c4-9a0b-d3238750b08f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:37.278745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476ead-d1c9-48c4-9a0b-d3238750b08f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:37.278861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476ead-d1c9-48c4-9a0b-d3238750b08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:37.298937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e2476ead-d1c9-48c4-9a0b-d3238750b08f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:37.300142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e2476ead-d1c9-48c4-9a0b-d3238750b08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:37.308548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:37.343516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd30e2-33f6-4dc8-87c0-7d0bceb1544a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:37.354969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd30e2-33f6-4dc8-87c0-7d0bceb1544a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:37.358494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047f63d-b738-4d3a-8077-0b3e7ab2b39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:37.359086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047f63d-b738-4d3a-8077-0b3e7ab2b39e service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:37.359373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047f63d-b738-4d3a-8077-0b3e7ab2b39e service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:37.359646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047f63d-b738-4d3a-8077-0b3e7ab2b39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:37.369266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047f63d-b738-4d3a-8077-0b3e7ab2b39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:37.370702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047f63d-b738-4d3a-8077-0b3e7ab2b39e service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:37.379893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 762/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:41.064683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032990ea-764f-479e-b70e-032e26bff9b0 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:41.065290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032990ea-764f-479e-b70e-032e26bff9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:41.065290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032990ea-764f-479e-b70e-032e26bff9b0 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:41.066677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:41.072929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3a4ef-021a-4b4e-a313-36209468a2f6 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:41.073391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3a4ef-021a-4b4e-a313-36209468a2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:41.073531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3a4ef-021a-4b4e-a313-36209468a2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:41.092940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a8a3a4ef-021a-4b4e-a313-36209468a2f6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:41.094177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a8a3a4ef-021a-4b4e-a313-36209468a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:41.101978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:41.412656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032990ea-764f-479e-b70e-032e26bff9b0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:41.415012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-032990ea-764f-479e-b70e-032e26bff9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:41.479420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:41.492778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032990ea-764f-479e-b70e-032e26bff9b0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:41.496559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb20f9-8781-4c6f-a779-bc1e082ccb18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:41.497202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb20f9-8781-4c6f-a779-bc1e082ccb18 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:41.497485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb20f9-8781-4c6f-a779-bc1e082ccb18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:41.497758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb20f9-8781-4c6f-a779-bc1e082ccb18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:41.506525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb20f9-8781-4c6f-a779-bc1e082ccb18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:41.508628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb20f9-8781-4c6f-a779-bc1e082ccb18 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:41.518074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 773/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:43.626302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec0791-2827-4dac-a628-fb1fd38cdb85 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.626833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec0791-2827-4dac-a628-fb1fd38cdb85 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.626833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec0791-2827-4dac-a628-fb1fd38cdb85 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.628228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:23:43.634008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bee07-1b47-4305-93d4-888e6c50643a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.634431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bee07-1b47-4305-93d4-888e6c50643a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.634539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bee07-1b47-4305-93d4-888e6c50643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.655842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-579bee07-1b47-4305-93d4-888e6c50643a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:43.657328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-579bee07-1b47-4305-93d4-888e6c50643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:43.665265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:43.966773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec0791-2827-4dac-a628-fb1fd38cdb85 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.968444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1dec0791-2827-4dac-a628-fb1fd38cdb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:43.981549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec0791-2827-4dac-a628-fb1fd38cdb85 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.985094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6978d-25e1-4212-bb62-8bb5bf0fb221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.985696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6978d-25e1-4212-bb62-8bb5bf0fb221 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.985976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6978d-25e1-4212-bb62-8bb5bf0fb221 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.986255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6978d-25e1-4212-bb62-8bb5bf0fb221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.995185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6978d-25e1-4212-bb62-8bb5bf0fb221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.996599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6978d-25e1-4212-bb62-8bb5bf0fb221 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:44.005924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 765/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:53.648894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=11 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:54.001857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.005752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3f355-8927-4fb4-b251-63ab374d9f5e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.006363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3f355-8927-4fb4-b251-63ab374d9f5e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.006363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3f355-8927-4fb4-b251-63ab374d9f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.031053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-11c3f355-8927-4fb4-b251-63ab374d9f5e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.032743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-11c3f355-8927-4fb4-b251-63ab374d9f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.040978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4d39d-c858-4329-acd1-154e44c19065 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.042128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4d39d-c858-4329-acd1-154e44c19065 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.042584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4d39d-c858-4329-acd1-154e44c19065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.051730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 775/1540] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.073272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.073557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.073839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.086361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5fe4d39d-c858-4329-acd1-154e44c19065 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.087939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5fe4d39d-c858-4329-acd1-154e44c19065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.113808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.115038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.115672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 766/1541] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.133527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd2ef4-12b3-45f4-be2b-2fbc489c651a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.134272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd2ef4-12b3-45f4-be2b-2fbc489c651a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.134711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd2ef4-12b3-45f4-be2b-2fbc489c651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.136361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 776/1542] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:10.150565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.169059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-70cd2ef4-12b3-45f4-be2b-2fbc489c651a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.171000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-70cd2ef4-12b3-45f4-be2b-2fbc489c651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.174061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379e818-5846-4cce-b610-41d63e1963aa None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.179648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 767/1543] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:10.181220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.181610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.182384 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.187422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.188044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.188326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.188604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.189287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:10.189590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.198315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.230793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.237294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.242065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 777/1544] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.243048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.276556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 768/1545] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.282770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.286924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c79ce-b3a4-49fb-80d1-ec1c0327eff6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.304069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5b048-81c2-4ae1-97f4-bb9442a6ab71 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.315548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.315548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.315548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.315548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.316056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:10.316056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.322825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adcee6-6193-412e-836e-51615b0efe16 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.327424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41cd1-7b18-451b-8aad-56ecd6659c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.328730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41cd1-7b18-451b-8aad-56ecd6659c7f admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.329271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41cd1-7b18-451b-8aad-56ecd6659c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.329768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41cd1-7b18-451b-8aad-56ecd6659c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.330627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae41cd1-7b18-451b-8aad-56ecd6659c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.338271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae41cd1-7b18-451b-8aad-56ecd6659c7f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.365035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 778/1546] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.377055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ee928-1eba-4e4a-b55c-34f74abb760b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.377055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ee928-1eba-4e4a-b55c-34f74abb760b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.377055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ee928-1eba-4e4a-b55c-34f74abb760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.396827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1e7ee928-1eba-4e4a-b55c-34f74abb760b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.399061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1e7ee928-1eba-4e4a-b55c-34f74abb760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.409163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 779/1547] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.410849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b63cb-9204-41ea-a43e-1730f4d6882b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.411339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b63cb-9204-41ea-a43e-1730f4d6882b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.411814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b63cb-9204-41ea-a43e-1730f4d6882b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.443624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-de4b63cb-9204-41ea-a43e-1730f4d6882b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.444822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-de4b63cb-9204-41ea-a43e-1730f4d6882b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.457874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 780/1548] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.457874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.457874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.457874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.483160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 769/1549] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/projects => generated 393 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:10.484441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0862f8-b134-4a44-8417-0c185a5fd7b9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.484733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0862f8-b134-4a44-8417-0c185a5fd7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.484974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.485333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0862f8-b134-4a44-8417-0c185a5fd7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.486477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.508624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ef0862f8-b134-4a44-8417-0c185a5fd7b9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.509918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ef0862f8-b134-4a44-8417-0c185a5fd7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.550848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 781/1550] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.551591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.561528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6770b-b8e0-4547-9f69-303fde9f752e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.566117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.566717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.566999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.567276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.567904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.568914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:25:10.568914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:25:10.568914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:25:10.569918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 770/1551] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.570136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be86e-9e64-4035-bd7b-2f8cff3f24d2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.571726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bf4ca-5ff1-4a14-b960-54b725f63e5e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.572016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bf4ca-5ff1-4a14-b960-54b725f63e5e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.572295 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bf4ca-5ff1-4a14-b960-54b725f63e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.592563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-cf4bf4ca-5ff1-4a14-b960-54b725f63e5e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.597589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cf4bf4ca-5ff1-4a14-b960-54b725f63e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.601882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 771/1552] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.603104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.603390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.603661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.623002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.624217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.632568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 772/1553] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.633556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.643630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c520535-8717-4404-bd77-b230f62b0e4f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.644470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 782/1554] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:10.651490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74af63-e16d-4c42-9268-de1a2ab91991 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.651490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74af63-e16d-4c42-9268-de1a2ab91991 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.651490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74af63-e16d-4c42-9268-de1a2ab91991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.670496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cf74af63-e16d-4c42-9268-de1a2ab91991 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.671693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cf74af63-e16d-4c42-9268-de1a2ab91991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.709127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.710156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.710438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.710712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.711423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:10.712085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.713582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:25:10.713582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:25:10.713582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:25:10.714800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.727792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 773/1555] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.728577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.731015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 783/1556] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.732276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0066cf6-1c39-4e1a-84b5-975a206a7dbf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.732558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0066cf6-1c39-4e1a-84b5-975a206a7dbf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.732844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0066cf6-1c39-4e1a-84b5-975a206a7dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.746192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e880f-6801-43ba-9019-5d9b66995d79 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.752382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.752981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.753246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e0066cf6-1c39-4e1a-84b5-975a206a7dbf None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.753348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.753556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.754173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.754482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0066cf6-1c39-4e1a-84b5-975a206a7dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.755176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:25:10.755176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:25:10.755176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:25:10.756382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.762628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 784/1557] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.763799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.764083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.764357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.783179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.784370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.792796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 785/1558] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.793586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.802562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e19ee-fb78-4e92-84b7-ff10b07e5af9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.808114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.808728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.809032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.809321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.810012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:10.810359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.811983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e279-2ff5-4b5f-b47a-02f3f95a801a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.821134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.827050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 786/1559] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.827050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.827050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.827050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.842727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 774/1560] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:10.845932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.846745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.847206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.855710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 787/1561] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:10.856463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.858210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ace4ca-0e41-4468-87ea-ade0e2649eab admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.863226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa70e6-bfd8-4a41-96ea-6823d6b97500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.863902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa70e6-bfd8-4a41-96ea-6823d6b97500 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.864180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa70e6-bfd8-4a41-96ea-6823d6b97500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.864448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa70e6-bfd8-4a41-96ea-6823d6b97500 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.865103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa70e6-bfd8-4a41-96ea-6823d6b97500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.866502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda95f-b62f-4b0a-b122-caf1d7d9b417 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.867087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa70e6-bfd8-4a41-96ea-6823d6b97500 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.872256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.872856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.873174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.873449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.874075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.875974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.880181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.956346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 775/1562] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:10.957631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.957978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.958278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.978803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.980047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.980961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 788/1563] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:10.985187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.041711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 776/1564] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:11.042641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.046464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458375a5-8f4c-4f09-976e-e51169b89ea8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.051929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc583c00-7525-4f52-9829-75f120a02092 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.052221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.052929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.053381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.053841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.055667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.055667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.057457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.058057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.058340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.058519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.058691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.059255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.059561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.061328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.072269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 789/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:11.073217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.081206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 777/1566] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:11.081206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.084304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a37a8e-b1eb-4098-aa7b-4f97b36d9045 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.089282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f925b2a-7dfa-48f6-aa8e-f29bd627ffb6 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.090002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.090589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.090866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.091131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.091745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.093749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.093982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.094573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.094856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.095134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.095778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.097424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.109454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 778/1567] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:11.110245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.120177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70a80-d6ff-4031-9aee-46e3c9cec857 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.124934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.125538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.125924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.126202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.126862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.127204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.128844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.141270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 779/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:11.142068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.152544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334e887-4aa2-4822-9c4a-419072a5cb81 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.157180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.158027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.158305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.158580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.158682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 790/1569] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.159397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.162327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.163949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.174817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 780/1570] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:11.175839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.185835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddb220-4bed-44b1-a435-289516096209 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.222306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7594f-7a45-49ae-b7bd-4fb177432279 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.227935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.228750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.228871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.229099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.229721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.231684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.244936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.245585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.245862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.246183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.246817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.247117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.248717 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.261179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 781/1571] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:11.262076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.271461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c6799-1926-443f-b26d-395627d2d059 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.276863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.277487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.277770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.278045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.278657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.280543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.294501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 791/1572] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.295360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.305856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a08ec9-3e2e-49b7-9aa9-373e94ecf2eb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.310335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.311029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.311313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.311583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.312194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.314116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.375049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 782/1573] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.378436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.396524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.401312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 792/1574] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.401312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.412368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.438936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188a5c7-1667-4efd-8a53-1b06248e9df7 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.460803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e398eb-b044-419c-9113-14b3f9173dd8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.466270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.466879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.467158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.467431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.468086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.470032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.499319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.499926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.500203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.500473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.501150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.502740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.518437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 783/1575] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:11.529285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.530349 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e8d50-9200-4782-8c5a-9eacf7ad0d5d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.535705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.536308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.536591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.536856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.537502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.539975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.553024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 793/1576] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.553849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.614950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69ece-b78a-44f4-a398-dbe1020cbf0d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.619546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.620153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.620434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.620706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.621381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.622990 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.635240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 794/1577] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:11.641146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.641302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 784/1578] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.641302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.646685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c853f0e-d70c-43b7-b2af-4ae0c6ab6587 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.655739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.655739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.655739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.655739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.656233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe68c40-cf1d-4f3d-9dc7-84ca89d2fc78 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.656328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.656328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.661703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.662559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.662840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.663120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.663818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.665968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.678400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 785/1579] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:11.679430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.690430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7671921-0de7-49bb-92bd-ca674a9d24d3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.695840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.697828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.701537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.701537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.701537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.701537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.745880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 795/1580] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.746812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.786071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 786/1581] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.787283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.798275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cec087-36d7-4be2-9cc4-fec2cfeb9b9b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.814835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f4f4-f516-418f-9228-199c5e409d68 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.819732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.821145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.821434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.821702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.822326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.824944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.865915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.866506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.866791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.867068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] PATH_INFO: `/v3/projects/c0c9fae66a6f4ac9a0c1b0629e597906/users/4dfd3ee79da943b189824630b5c393ee/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.885128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c9fae66a6f4ac9a0c1b0629e597906, user_id=4dfd3ee79da943b189824630b5c393ee, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.888797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.906186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.912033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 796/1582] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/users => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:11.913035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.923240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac2ca6-326b-4f08-8a2b-48e2c75d0833 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.929227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.930917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.931187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.931460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.932126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.934045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.981139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.985818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 787/1583] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:11 2022] PUT /identity/v3/projects/c0c9fae66a6f4ac9a0c1b0629e597906/users/4dfd3ee79da943b189824630b5c393ee/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:11.986823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.009732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 797/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:11 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:12.010571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.020535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfcc24-f880-4e54-9600-656da35e85d5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.063879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0c69-607c-4631-98e3-4e4f1fd03ff1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.068684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.069386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.069667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.069979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] PATH_INFO: `/v3/projects/f3070abbdec34c779d4bd382b2bb1697/users/0ace2374550e40ce812407406c262ac5/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.081234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.081860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.083621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.083621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.083621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.084637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.085536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3070abbdec34c779d4bd382b2bb1697, user_id=0ace2374550e40ce812407406c262ac5, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.089065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.098128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 798/1585] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.098978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.109873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356adc8-3d99-47fb-ba30-9d49cab6ba36 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.116357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.116875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.117194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.118102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.118750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.120402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.133956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 799/1586] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.133956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.142849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc1831-9fe2-41f4-bbc6-2a9b55a97bb5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.147509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.148107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.148383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.149884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.150517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.152422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.170450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 788/1587] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:11 2022] PUT /identity/v3/projects/f3070abbdec34c779d4bd382b2bb1697/users/0ace2374550e40ce812407406c262ac5/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:12.175713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.255903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f0463-2397-4db4-ab02-b1f8652c8d02 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.261586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a9fe0-6b31-4b09-b036-512ed56397e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.262206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9fe0-6b31-4b09-b036-512ed56397e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.262489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9fe0-6b31-4b09-b036-512ed56397e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.262769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9fe0-6b31-4b09-b036-512ed56397e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.263414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a9fe0-6b31-4b09-b036-512ed56397e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.265112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a9fe0-6b31-4b09-b036-512ed56397e8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.267829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 800/1588] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:12 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:12.268709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.284666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 789/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.284666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.285005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21e3ee-957d-4a97-bf5c-bdc6b79c5992 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.285174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.285174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.290260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.291007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.291475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.291933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.292969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.299468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.307026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:12.308258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:12.316600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 790/1590] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:12.322011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.322742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 801/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.322742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.331137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb328b-7a19-4fc7-ab4f-99a4e78f8ff6 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.335169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65c855-53b1-455a-a76b-277fe54e8fe4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.338282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.338902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.339223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.339451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.340102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.341327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef824886-f48f-4ffa-8109-1c16b78d3d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.341938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef824886-f48f-4ffa-8109-1c16b78d3d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.342132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.342246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef824886-f48f-4ffa-8109-1c16b78d3d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.343361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef824886-f48f-4ffa-8109-1c16b78d3d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.343361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef824886-f48f-4ffa-8109-1c16b78d3d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.344887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef824886-f48f-4ffa-8109-1c16b78d3d44 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.353320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 791/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.361086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.361290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 802/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.361290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.361290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.361290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.367070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44371fde-5b85-4f2d-b1d6-e7368f4dbe8c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.374336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.374995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.375283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.375545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] PATH_INFO: `/v3/projects/e2d9f797cd07444f9fe76043e0b9c33d/users/8d3dd4ce85b24f5a9e57dc7c4abd4bf7/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.381563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:12.383415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:12.391372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 803/1594] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:12.392194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.393869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d9f797cd07444f9fe76043e0b9c33d, user_id=8d3dd4ce85b24f5a9e57dc7c4abd4bf7, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.396847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.409194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324eb95d-8b4b-4358-b970-8457f8b3fe39 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.419694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.420311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.420591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.421882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.421882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.423193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.437398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 804/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.438237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.450590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285ebc-7f0b-4b19-9968-382280ed08d2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.458402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.459419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.459944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.460450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.461976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.463579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.473762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 805/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.477249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.480467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 792/1597] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:12 2022] PUT /identity/v3/projects/e2d9f797cd07444f9fe76043e0b9c33d/users/8d3dd4ce85b24f5a9e57dc7c4abd4bf7/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:12.481395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.488527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c81d4-d9df-4651-bf50-7381c0f9ea5f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.523502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:12.543109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b3222-97d3-4abe-a351-bbb1f047d74a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.546855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.547486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.547762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.548032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.556880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.557530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.557814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.558092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.558930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.560332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.561757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:12.571694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 806/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.572542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.582117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0c10b-94be-4de2-a5f1-abc7a30947d2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.587682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.591693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.591693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.591693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] PATH_INFO: `/v3/projects/d63f4e449ad24501aab056be873063fb/users/abdfc175c6714dd6929a165693c8c07e/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.603912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63f4e449ad24501aab056be873063fb, user_id=abdfc175c6714dd6929a165693c8c07e, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.605234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:25:12.605234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:25:12.605234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:25:12.607357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.611311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.612743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.624164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 793/1599] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:12.624164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.632508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2f0b1-0ff2-4ee1-8707-b354835fc677 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.637782 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.638376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.638644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.638914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.639558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.641161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.652146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 794/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.654838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.663178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43f557-1879-49a1-bc86-e7fde7845b75 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.671291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.671291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.671291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.671291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.671646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 807/1601] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:12 2022] PUT /identity/v3/projects/d63f4e449ad24501aab056be873063fb/users/abdfc175c6714dd6929a165693c8c07e/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:12.672386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.677444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.678865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.689405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 795/1602] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:12.690236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.700484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1d65e-0271-496e-88d6-f726eef7006a service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.763547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055cb07-5a1c-430f-8872-a193673d2c8c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.766636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.767301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.767625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.767904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] PATH_INFO: `/v3/projects/e52ecd30866b4733acd28c43adbc29a0/users/a46db4eba75f4c49a394bffe4f220fe0/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.772188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad6fce-bf19-488c-a827-e80db2fcd160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.773037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad6fce-bf19-488c-a827-e80db2fcd160 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.778087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad6fce-bf19-488c-a827-e80db2fcd160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.778087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad6fce-bf19-488c-a827-e80db2fcd160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.778087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad6fce-bf19-488c-a827-e80db2fcd160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.778087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad6fce-bf19-488c-a827-e80db2fcd160 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.784753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52ecd30866b4733acd28c43adbc29a0, user_id=a46db4eba75f4c49a394bffe4f220fe0, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.787449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.792495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 808/1603] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:12.792495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.792495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.792495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.827336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] MFA Rules not processed for user `8d3dd4ce85b24f5a9e57dc7c4abd4bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:12.828647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:12.861763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 796/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:12 2022] PUT /identity/v3/projects/e52ecd30866b4733acd28c43adbc29a0/users/a46db4eba75f4c49a394bffe4f220fe0/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:12.861763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.903934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 809/1605] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:12.904644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:12.917783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:12.917972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c5c91-182d-49a6-9bfc-90a92899b291 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.933862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc58c-1582-41a4-a0a2-7bedc8942eb5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:12.939524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba7478-5ed2-474f-9f33-1a2319eb1005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.939907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7478-5ed2-474f-9f33-1a2319eb1005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.940190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7478-5ed2-474f-9f33-1a2319eb1005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.940463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7478-5ed2-474f-9f33-1a2319eb1005 admin admin] PATH_INFO: `/v3/projects/23f7ceb53f794eb39f7cb346bdebcfdf/users/b1b736ca1270429190b76b6ea953ca67/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:12.960442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba7478-5ed2-474f-9f33-1a2319eb1005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f7ceb53f794eb39f7cb346bdebcfdf, user_id=b1b736ca1270429190b76b6ea953ca67, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:12.963540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba7478-5ed2-474f-9f33-1a2319eb1005 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:12.974956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:12.981989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:12.982666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:12.982946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:12.983288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] PATH_INFO: `/v3/projects/affff79fbec84e6f9a3714af610efedc/users/1229aa59a9964464ba072d9b9c5227a0/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.000150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=affff79fbec84e6f9a3714af610efedc, user_id=1229aa59a9964464ba072d9b9c5227a0, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.002957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.061843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 797/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:12 2022] PUT /identity/v3/projects/23f7ceb53f794eb39f7cb346bdebcfdf/users/b1b736ca1270429190b76b6ea953ca67/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:13.066658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7ff0-51c7-4c47-8bc3-90dc21d78444 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.067236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7ff0-51c7-4c47-8bc3-90dc21d78444 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.067236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7ff0-51c7-4c47-8bc3-90dc21d78444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.093934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 810/1607] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:12 2022] PUT /identity/v3/projects/affff79fbec84e6f9a3714af610efedc/users/1229aa59a9964464ba072d9b9c5227a0/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:13.093934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.112081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-513f7ff0-51c7-4c47-8bc3-90dc21d78444 None None] MFA Rules not processed for user `abdfc175c6714dd6929a165693c8c07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:13.113047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-513f7ff0-51c7-4c47-8bc3-90dc21d78444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.185310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4e17b-51fc-454f-a9eb-729fde1ca350 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.188978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a39142-4d3a-4845-aa7e-ee335175097a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.189802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39142-4d3a-4845-aa7e-ee335175097a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.190198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39142-4d3a-4845-aa7e-ee335175097a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.190746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39142-4d3a-4845-aa7e-ee335175097a admin admin] PATH_INFO: `/v3/projects/26a97b47cc804e598c5e8680d279de19/users/c9577bc4c4c546ca9fc220ebd3a16346/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.194746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 798/1608] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:13.196447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.196892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.197197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.213539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a39142-4d3a-4845-aa7e-ee335175097a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a97b47cc804e598c5e8680d279de19, user_id=c9577bc4c4c546ca9fc220ebd3a16346, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.219914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a39142-4d3a-4845-aa7e-ee335175097a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.224089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:13.225637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.233501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 799/1609] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:13.235257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.244283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891d9bd-33de-49e8-90f1-f1a924cc60e4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.248903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364354c-a54a-4433-8837-ea837712e3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.249535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364354c-a54a-4433-8837-ea837712e3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.249814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364354c-a54a-4433-8837-ea837712e3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.250082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364354c-a54a-4433-8837-ea837712e3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.250704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364354c-a54a-4433-8837-ea837712e3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.252637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364354c-a54a-4433-8837-ea837712e3e0 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.329465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 811/1610] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:13 2022] PUT /identity/v3/projects/26a97b47cc804e598c5e8680d279de19/users/c9577bc4c4c546ca9fc220ebd3a16346/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:13.330759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fd30f-51ea-4ad2-a9d2-2b095895fcb0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.331045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fd30f-51ea-4ad2-a9d2-2b095895fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.331322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fd30f-51ea-4ad2-a9d2-2b095895fcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.350640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 800/1611] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:13.350640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61695e4e-2255-416c-9ea2-6f483b3e302f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.350640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61695e4e-2255-416c-9ea2-6f483b3e302f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.350640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61695e4e-2255-416c-9ea2-6f483b3e302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.356959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-419fd30f-51ea-4ad2-a9d2-2b095895fcb0 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:13.359077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-419fd30f-51ea-4ad2-a9d2-2b095895fcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.384220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-61695e4e-2255-416c-9ea2-6f483b3e302f None None] MFA Rules not processed for user `1229aa59a9964464ba072d9b9c5227a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:13.385608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-61695e4e-2255-416c-9ea2-6f483b3e302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.459246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 812/1612] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:13.461736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565307-7dee-43ce-8611-e55c2792a301 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.462061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565307-7dee-43ce-8611-e55c2792a301 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.462443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565307-7dee-43ce-8611-e55c2792a301 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.463961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 813/1613] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 10 13:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:13.464531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565307-7dee-43ce-8611-e55c2792a301 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.467419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-41565307-7dee-43ce-8611-e55c2792a301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.493728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 801/1614] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:13.495380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.495579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.495973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.517556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:13.518953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.530014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 802/1615] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:13.530961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.545871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565307-7dee-43ce-8611-e55c2792a301 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.551951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb43ed-10b4-4cd3-9860-1607b0f119e3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.558943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.559469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.559771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.560024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.560663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.561789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.562461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.562737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.562858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.563048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.573903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.575469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.576862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:13.593825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 814/1616] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:13.595824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.611243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcb01-11f3-4ed6-8181-7b714376226a service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.616448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0352e-f3bc-49aa-92f0-c992c9cc6ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.617144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0352e-f3bc-49aa-92f0-c992c9cc6ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.617530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0352e-f3bc-49aa-92f0-c992c9cc6ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.617766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0352e-f3bc-49aa-92f0-c992c9cc6ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.626170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:13.630731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0352e-f3bc-49aa-92f0-c992c9cc6ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.633146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0352e-f3bc-49aa-92f0-c992c9cc6ac2 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.646634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 815/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:13.648067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.649768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.650206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.673099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 803/1618] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:13.674081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.685075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2df3-3b0c-4c1d-9874-7e8e43f8600c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.690713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.691764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.692155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.692534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.693333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.697809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.705542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:13.707658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:13.799743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 804/1619] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:13.799743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.801665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 816/1620] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:13.803281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.816416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1413f-df89-49b5-a569-6452f6f388f8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.867367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cece-866f-4dc3-b0ab-d5296ae5f369 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.872164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.873366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.873823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.874441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.887321 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.888275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.889225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.889225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.890023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.892274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.909362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 817/1621] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:13.909362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:13.922796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a397e-94cb-43ab-837c-859bda887b7a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:13.931091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:13.931911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:13.932245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:13.932668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:13.934503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.936182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.969830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:13.971450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:13.982230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 805/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:13.983050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.034658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 818/1623] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:13 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:14.035663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.050197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd007-e3d4-4f9f-8c61-7dfad7a48468 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.057237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.058051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.058477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.058849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] PATH_INFO: `/v3/projects/fb6b6db7413f41d2bdc092b5d1c285b2/users/eafe34c557a244a5b3b8318257383c6d/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.075304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6b6db7413f41d2bdc092b5d1c285b2, user_id=eafe34c557a244a5b3b8318257383c6d, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.079215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.079758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739acf5-2ccf-45c6-92a4-cafb9d88334f service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.086409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.087799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.087799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.088005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.124238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:14.166336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 819/1624] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:14 2022] PUT /identity/v3/projects/fb6b6db7413f41d2bdc092b5d1c285b2/users/eafe34c557a244a5b3b8318257383c6d/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:14.167358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.176776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.178422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.193567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 806/1625] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:14.194821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.210044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f8e8b-8af1-42fc-a5d9-4fb0341a26ab service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.213498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:14.265449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56233-2177-4ba9-b65b-5ea2bde8b95c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.278177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.278998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.279455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.279909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.280721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.282779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.293224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.294123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.294479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.294866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.295615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.299111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 820/1626] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:14.299262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.300406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.314409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc9f8f-6a7d-481a-af03-84afae378c40 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.314823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 807/1627] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:14.318566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.320729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.321557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.321886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.322127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] PATH_INFO: `/v3/projects/0df16b9916ef4225919e0371343d3e4b/users/53fd9d3e6c6641f3ba766eb78af931c3/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.340778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28018c2-de11-4b4e-ba31-8c7820157dbf admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.347375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a22a5-e2ed-411f-9919-691a1f5f6364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.348368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a22a5-e2ed-411f-9919-691a1f5f6364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.349459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df16b9916ef4225919e0371343d3e4b, user_id=53fd9d3e6c6641f3ba766eb78af931c3, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.349705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a22a5-e2ed-411f-9919-691a1f5f6364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.350254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a22a5-e2ed-411f-9919-691a1f5f6364 admin admin] PATH_INFO: `/v3/projects/fb6b6db7413f41d2bdc092b5d1c285b2/users/eafe34c557a244a5b3b8318257383c6d/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.352588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a22a5-e2ed-411f-9919-691a1f5f6364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6b6db7413f41d2bdc092b5d1c285b2, user_id=eafe34c557a244a5b3b8318257383c6d, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.353021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.356104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a22a5-e2ed-411f-9919-691a1f5f6364 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.368178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 808/1628] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:14 2022] PUT /identity/v3/projects/fb6b6db7413f41d2bdc092b5d1c285b2/users/eafe34c557a244a5b3b8318257383c6d/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:14.379593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.379957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.380244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.452797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] MFA Rules not processed for user `eafe34c557a244a5b3b8318257383c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:14.454586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:14.455557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 821/1629] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:14 2022] PUT /identity/v3/projects/0df16b9916ef4225919e0371343d3e4b/users/53fd9d3e6c6641f3ba766eb78af931c3/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:14.459366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.605477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 809/1630] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:14.614634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.618449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ef02-4ac7-4fce-bfdf-178c93832a68 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.626148 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252878a-3eb0-4f41-8589-9d05af290368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.639489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252878a-3eb0-4f41-8589-9d05af290368 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.640064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252878a-3eb0-4f41-8589-9d05af290368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.641049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252878a-3eb0-4f41-8589-9d05af290368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.642168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252878a-3eb0-4f41-8589-9d05af290368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.644670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252878a-3eb0-4f41-8589-9d05af290368 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.660488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 822/1631] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:25:14.663368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938da25-7453-4c70-b6ef-f9d9d6963973 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.663819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938da25-7453-4c70-b6ef-f9d9d6963973 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.668037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938da25-7453-4c70-b6ef-f9d9d6963973 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.668037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 823/1632] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 10 13:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:14.669026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938da25-7453-4c70-b6ef-f9d9d6963973 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.706408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938da25-7453-4c70-b6ef-f9d9d6963973 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.713718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466ebbc-808e-48b1-8e95-5a71646f1960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.714450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466ebbc-808e-48b1-8e95-5a71646f1960 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.714729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466ebbc-808e-48b1-8e95-5a71646f1960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.715301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466ebbc-808e-48b1-8e95-5a71646f1960 admin admin] PATH_INFO: `/v3/projects/0df16b9916ef4225919e0371343d3e4b/users/53fd9d3e6c6641f3ba766eb78af931c3/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.718061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466ebbc-808e-48b1-8e95-5a71646f1960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df16b9916ef4225919e0371343d3e4b, user_id=53fd9d3e6c6641f3ba766eb78af931c3, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.721697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466ebbc-808e-48b1-8e95-5a71646f1960 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.750285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 824/1633] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:14 2022] PUT /identity/v3/projects/0df16b9916ef4225919e0371343d3e4b/users/53fd9d3e6c6641f3ba766eb78af931c3/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:14.750285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9684a-655a-425d-ad11-dfd63b2383cf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.750285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9684a-655a-425d-ad11-dfd63b2383cf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.750285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9684a-655a-425d-ad11-dfd63b2383cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.771166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdea8a-188f-40da-bd0c-93a6c7a62e95 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.773153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4ae9684a-655a-425d-ad11-dfd63b2383cf None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:14.775003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4ae9684a-655a-425d-ad11-dfd63b2383cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:14.864587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.867859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.867859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.869337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.892696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 825/1634] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:14.894087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.896344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.896642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.946790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] MFA Rules not processed for user `53fd9d3e6c6641f3ba766eb78af931c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:14.948155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:14.981448 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.983486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.997368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 810/1635] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:25:14.999158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.001437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.018872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5c9b9-9c11-49dc-8e70-24ad3cbab48a service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.033373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.036560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.036940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.039768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.055715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 826/1636] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:15.063136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.064752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.076643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.078575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 811/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:15.091029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24283299-39b7-47ef-bd2a-6f711cda101d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.096519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0869f8-3f58-4c57-9eff-c0ef6f11fad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.097044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0869f8-3f58-4c57-9eff-c0ef6f11fad9 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.097499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0869f8-3f58-4c57-9eff-c0ef6f11fad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.097849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0869f8-3f58-4c57-9eff-c0ef6f11fad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.110715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0869f8-3f58-4c57-9eff-c0ef6f11fad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.113340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0869f8-3f58-4c57-9eff-c0ef6f11fad9 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.125709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 827/1638] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:15.231979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.642930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.652852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270583-35bf-4f33-9592-738526c2b845 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.665371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ef3d0-be25-44be-99d4-3f0a4f9e3898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.666270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ef3d0-be25-44be-99d4-3f0a4f9e3898 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.667125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ef3d0-be25-44be-99d4-3f0a4f9e3898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.667847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ef3d0-be25-44be-99d4-3f0a4f9e3898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.797784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ef3d0-be25-44be-99d4-3f0a4f9e3898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.799248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ef3d0-be25-44be-99d4-3f0a4f9e3898 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.818900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 812/1639] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:16.981979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.981979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.981979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.981979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] MFA Rules not processed for user `b1b736ca1270429190b76b6ea953ca67`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:16.981979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:16.984427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca7c0-5187-41d3-b600-0839f934d378 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.984427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca7c0-5187-41d3-b600-0839f934d378 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.984427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca7c0-5187-41d3-b600-0839f934d378 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.984427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 813/1640] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 10 13:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:16.984427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed627-7981-4e0e-95ac-5d84d44a3363 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.984427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed627-7981-4e0e-95ac-5d84d44a3363 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.986676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed627-7981-4e0e-95ac-5d84d44a3363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.025666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-09aed627-7981-4e0e-95ac-5d84d44a3363 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:17.027173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-09aed627-7981-4e0e-95ac-5d84d44a3363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:17.047534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 814/1641] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 10 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:17.056387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5151d-c24a-4dc9-8c89-66b4458321e6 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.056387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5151d-c24a-4dc9-8c89-66b4458321e6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.056387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5151d-c24a-4dc9-8c89-66b4458321e6 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.057713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 815/1642] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 10 13:25:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:17.060342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.061576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.064276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.109850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:17.111297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:17.130262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 828/1643] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:17.131275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:17.137825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 816/1644] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:25:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:17.240563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:17.253467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec494b-5723-44cd-a1b8-c20d0eabedb4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:17.258364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecc5dc-2701-453e-aff9-d5997418d2e5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:17.263354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:17.264001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.265308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.265602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.266268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:17.268598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:17.270631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:17.270926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.271530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.271530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.298649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:17.298649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:17.298649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:17.320182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 817/1645] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:25:17.415944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:17.433540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 829/1646] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:17 2022] POST /identity/v3/projects => generated 403 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:17.436278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:17.461439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af6c0-03f0-49f0-bcbe-dfa5d506c1d1 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:17.578237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:17.579077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.579461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.579886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.580709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:17.583441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:17.685804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 818/1647] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:17 2022] POST /identity/v3/users => generated 308 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:17.691688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:17.708602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601efb21-9f04-492c-af69-0be3e948c589 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:17.727048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:17.728761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.729368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.729655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.733133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:17.736974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:17.771051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 830/1648] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:25:17.772469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:17.788060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e547f-5c77-43b0-bfb1-778d4bd2feb8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:17.802078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:17.802954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.803243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.803535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] PATH_INFO: `/v3/projects/eaf62c61a39a4de9b8f828db5ff749cb/users/b7c209a2a4dc481ab14d587cbcaea91e/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.832428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:17.852314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7ff7-c533-460a-a172-398e1ac23cfb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:17.865579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf62c61a39a4de9b8f828db5ff749cb, user_id=b7c209a2a4dc481ab14d587cbcaea91e, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:17.865763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:17.866505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:17.869984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:17.870855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:17.871966 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] PATH_INFO: `/v3/users/1229aa59a9964464ba072d9b9c5227a0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:17.876603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1229aa59a9964464ba072d9b9c5227a0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:17.879524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:25:17.879524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Aug 10 13:25:17.879524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: Aug 10 13:25:17.884434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:17.948513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 819/1649] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:17 2022] PUT /identity/v3/projects/eaf62c61a39a4de9b8f828db5ff749cb/users/b7c209a2a4dc481ab14d587cbcaea91e/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:17.952218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.052404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc99ba9-ba41-4540-9ce0-8daeae4b355b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.059960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.068646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.068646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.068646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.068646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:18.068646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:18.076397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 820/1650] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:18.081102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.101715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9658-137c-49d9-afb2-dca7d9796a4c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.101715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0502ce-3c50-4e61-a52e-234fa49aca5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.102827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0502ce-3c50-4e61-a52e-234fa49aca5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.103268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0502ce-3c50-4e61-a52e-234fa49aca5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.103733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0502ce-3c50-4e61-a52e-234fa49aca5f admin admin] PATH_INFO: `/v3/projects/eaf62c61a39a4de9b8f828db5ff749cb/users/b7c209a2a4dc481ab14d587cbcaea91e/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.107582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0502ce-3c50-4e61-a52e-234fa49aca5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf62c61a39a4de9b8f828db5ff749cb, user_id=b7c209a2a4dc481ab14d587cbcaea91e, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:18.113141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0502ce-3c50-4e61-a52e-234fa49aca5f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:18.127957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 821/1651] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:18 2022] PUT /identity/v3/projects/eaf62c61a39a4de9b8f828db5ff749cb/users/b7c209a2a4dc481ab14d587cbcaea91e/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:18.156309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1229aa59a9964464ba072d9b9c5227a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:18.188302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1229aa59a9964464ba072d9b9c5227a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:18.296970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1229aa59a9964464ba072d9b9c5227a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:18.298629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1229aa59a9964464ba072d9b9c5227a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:18.342529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1229aa59a9964464ba072d9b9c5227a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:18.352201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 831/1652] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:17 2022] DELETE /identity/v3/users/1229aa59a9964464ba072d9b9c5227a0 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:18.356769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.357313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.357423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.386938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:18.387741 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:18.446128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:18.480667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 822/1653] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:18.516168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.517115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:18.692695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258643-41e0-4ec3-bdea-ae9bf57b6318 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.702832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c4e0e-6bf7-495a-81ff-31ee75ad2cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.704599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c4e0e-6bf7-495a-81ff-31ee75ad2cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.704835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c4e0e-6bf7-495a-81ff-31ee75ad2cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.705191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c4e0e-6bf7-495a-81ff-31ee75ad2cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.730082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c4e0e-6bf7-495a-81ff-31ee75ad2cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:18.734047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c4e0e-6bf7-495a-81ff-31ee75ad2cd7 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:18.765046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 832/1654] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:20.326141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.330393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:20.361488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f3100-283d-4e3f-8baa-0accc0e6d88b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.368679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.370271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.372065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.372567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] PATH_INFO: `/v3/users/8d3dd4ce85b24f5a9e57dc7c4abd4bf7` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.377220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d3dd4ce85b24f5a9e57dc7c4abd4bf7)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.380831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.391216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8d36f-c9fc-4d23-b2de-7c74931c66ee None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.391772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8d36f-c9fc-4d23-b2de-7c74931c66ee None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.394616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8d36f-c9fc-4d23-b2de-7c74931c66ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.464456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-56f8d36f-c9fc-4d23-b2de-7c74931c66ee None None] MFA Rules not processed for user `b7c209a2a4dc481ab14d587cbcaea91e`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:20.466126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-56f8d36f-c9fc-4d23-b2de-7c74931c66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:20.580586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d3dd4ce85b24f5a9e57dc7c4abd4bf7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:20.670275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.687636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d3dd4ce85b24f5a9e57dc7c4abd4bf7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:20.701894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d3dd4ce85b24f5a9e57dc7c4abd4bf7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:20.703679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d3dd4ce85b24f5a9e57dc7c4abd4bf7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:20.735778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 833/1655] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:20.752876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d3dd4ce85b24f5a9e57dc7c4abd4bf7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:20.778756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 823/1656] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:20 2022] DELETE /identity/v3/users/8d3dd4ce85b24f5a9e57dc7c4abd4bf7 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:20.783901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.784362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.784774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.818564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:20.820134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:20.974851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 834/1657] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:21.069579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.071422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:21.253001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effb2dc-ac35-47ac-a3e1-ade59c01caa3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.262950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.263645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.263931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.264208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.287681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.290643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.306574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 824/1658] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:21.405529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.407004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:21.444010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1fec0-f3a2-4561-be50-a1e848ec5d23 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.461511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.463408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.463931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.464524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] PATH_INFO: `/v3/projects/affff79fbec84e6f9a3714af610efedc` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.473380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=affff79fbec84e6f9a3714af610efedc)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.480266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.698073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'affff79fbec84e6f9a3714af610efedc', 'request_id': 'req-81d0d2a6-2834-4040-badb-0708f77d2a6c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:21.713183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'affff79fbec84e6f9a3714af610efedc', 'request_id': 'req-81d0d2a6-2834-4040-badb-0708f77d2a6c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:21.723620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'affff79fbec84e6f9a3714af610efedc', 'request_id': 'req-81d0d2a6-2834-4040-badb-0708f77d2a6c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:21.819931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] The token cache is being invalidate because project affff79fbec84e6f9a3714af610efedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:21.820718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:21.823551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-81d0d2a6-2834-4040-badb-0708f77d2a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:21.846462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 835/1659] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:21 2022] DELETE /identity/v3/projects/affff79fbec84e6f9a3714af610efedc => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:21.850359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.869114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215683e-bcd8-4d6d-8934-a10e548d0538 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.073241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.092275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.093517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.094058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.095367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] PATH_INFO: `/v3/users/53fd9d3e6c6641f3ba766eb78af931c3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.097565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fd9d3e6c6641f3ba766eb78af931c3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.099770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.335100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fd9d3e6c6641f3ba766eb78af931c3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:22.453859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fd9d3e6c6641f3ba766eb78af931c3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:22.464356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fd9d3e6c6641f3ba766eb78af931c3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:22.466877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fd9d3e6c6641f3ba766eb78af931c3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:22.505867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ffdcd9aa-c3a9-45a7-8e6a-21c09acc474d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fd9d3e6c6641f3ba766eb78af931c3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:22.519243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 825/1660] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:21 2022] DELETE /identity/v3/users/53fd9d3e6c6641f3ba766eb78af931c3 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:22.798643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed3c1-aaf5-4c07-95c4-7ac4b3bd8cc3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.799134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed3c1-aaf5-4c07-95c4-7ac4b3bd8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.799330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed3c1-aaf5-4c07-95c4-7ac4b3bd8cc3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.800840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 836/1661] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:22.891014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.891551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.891761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.932755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:22.936985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:23.195705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 826/1662] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:23.227119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ed3c1-aaf5-4c07-95c4-7ac4b3bd8cc3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.227119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d72ed3c1-aaf5-4c07-95c4-7ac4b3bd8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:23.260039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ed3c1-aaf5-4c07-95c4-7ac4b3bd8cc3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.271952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.272765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.273757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.274343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.296139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.300647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.326525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 837/1663] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:23.987559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.989021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:24.130721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e29e-7f82-44f7-8c30-b158ceb7352d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.142404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.145778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.146335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.147002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] PATH_INFO: `/v3/projects/e2d9f797cd07444f9fe76043e0b9c33d` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.150290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d9f797cd07444f9fe76043e0b9c33d)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.152164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: WARNING py.warnings [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:25:24.152164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: warnings.warn(msg) Aug 10 13:25:24.152164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: Aug 10 13:25:24.160245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.214977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:24.390366 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d9f797cd07444f9fe76043e0b9c33d', 'request_id': 'req-e060bd69-da51-4529-8ee2-a038643685f8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:24.409048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d9f797cd07444f9fe76043e0b9c33d', 'request_id': 'req-e060bd69-da51-4529-8ee2-a038643685f8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:24.449800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d9f797cd07444f9fe76043e0b9c33d', 'request_id': 'req-e060bd69-da51-4529-8ee2-a038643685f8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:24.569023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] The token cache is being invalidate because project e2d9f797cd07444f9fe76043e0b9c33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:24.569135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:24.573278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:24.580758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 827/1664] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:23 2022] DELETE /identity/v3/projects/e2d9f797cd07444f9fe76043e0b9c33d => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:24.585047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.605311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14bf53-cb76-416e-a867-9fa7fe4ecd17 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.639964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.645702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:24.745795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.746607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.747001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.747383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] PATH_INFO: `/v3/users/eafe34c557a244a5b3b8318257383c6d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.749499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafe34c557a244a5b3b8318257383c6d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.751555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.759743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:24.789745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060bd69-da51-4529-8ee2-a038643685f8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.805416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.806229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.806564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.812184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] PATH_INFO: `/v3/users/b1b736ca1270429190b76b6ea953ca67` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.815133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b736ca1270429190b76b6ea953ca67)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.816678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.959575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafe34c557a244a5b3b8318257383c6d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:24.983973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafe34c557a244a5b3b8318257383c6d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.071874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafe34c557a244a5b3b8318257383c6d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.072736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafe34c557a244a5b3b8318257383c6d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.092285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafe34c557a244a5b3b8318257383c6d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.098009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b736ca1270429190b76b6ea953ca67'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.103511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 838/1665] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:24 2022] DELETE /identity/v3/users/eafe34c557a244a5b3b8318257383c6d => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:25.174445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.181413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:25.214399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b736ca1270429190b76b6ea953ca67'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.224399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b736ca1270429190b76b6ea953ca67'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.226945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b736ca1270429190b76b6ea953ca67'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.251484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e76f859-c7a2-4d3e-a1c9-34e4363d0a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b736ca1270429190b76b6ea953ca67'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.285449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 828/1666] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:24 2022] DELETE /identity/v3/users/b1b736ca1270429190b76b6ea953ca67 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:25.329924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f0dc3-cf10-4085-b77f-74fbe5744b7d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.433061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.433671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.433671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.433876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] PATH_INFO: `/v3/projects/0df16b9916ef4225919e0371343d3e4b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.436100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df16b9916ef4225919e0371343d3e4b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.436901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.606946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df16b9916ef4225919e0371343d3e4b', 'request_id': 'req-1b060390-5ff1-48ff-af6c-370c8d04fd2e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.630841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df16b9916ef4225919e0371343d3e4b', 'request_id': 'req-1b060390-5ff1-48ff-af6c-370c8d04fd2e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.668800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df16b9916ef4225919e0371343d3e4b', 'request_id': 'req-1b060390-5ff1-48ff-af6c-370c8d04fd2e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.786241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] The token cache is being invalidate because project 0df16b9916ef4225919e0371343d3e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:25.786538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.788262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b060390-5ff1-48ff-af6c-370c8d04fd2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:25.793458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 839/1667] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:25 2022] DELETE /identity/v3/projects/0df16b9916ef4225919e0371343d3e4b => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:25.798297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.798984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.799399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.837450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:25.840657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:25.900976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:25.951025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.100928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 829/1668] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:26.103704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.103996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.104280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.178920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:26.188474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:26.215810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 840/1669] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:26.218733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.236227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ce074-18b5-42c7-9fd6-cc063a2c1d8c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.249139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.249772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.250055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.250327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.251586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:26.251903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.253689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.295682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 830/1670] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:26.298595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.318320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac8f07-2ed0-4102-8d81-45dc16b3d705 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.324207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.325201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.325599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.326045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.326814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.328935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.495862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 841/1671] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:26 2022] POST /identity/v3/projects => generated 353 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:26.497350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.513965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dbb2e-723c-4d31-a70d-165e98e5781c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.639136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.641205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:26.690710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.691932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.696872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.697207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.698039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.710081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.874484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d4903-2289-4418-9b53-3496dcce0cfb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.897036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.897715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.899289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.899546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] PATH_INFO: `/v3/projects/23f7ceb53f794eb39f7cb346bdebcfdf` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.902933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 831/1672] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:26 2022] POST /identity/v3/users => generated 277 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:26.903905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f7ceb53f794eb39f7cb346bdebcfdf)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.906701 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.908281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.928282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aa063-04f4-47b1-a6f3-83782630c65e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.944089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.944709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.945588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.946041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.946883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.948726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.969030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 832/1673] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:26.972795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.983996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:27.018854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef7b81-d68d-4cdd-8228-e840e80517ad admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.037216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.037216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.037216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.037216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] PATH_INFO: `/v3/projects/fb6b6db7413f41d2bdc092b5d1c285b2` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.037810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6b6db7413f41d2bdc092b5d1c285b2)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.040751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.110747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f7ceb53f794eb39f7cb346bdebcfdf', 'request_id': 'req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.125691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f7ceb53f794eb39f7cb346bdebcfdf', 'request_id': 'req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.142296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f7ceb53f794eb39f7cb346bdebcfdf', 'request_id': 'req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.229221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6b6db7413f41d2bdc092b5d1c285b2', 'request_id': 'req-0a657054-a35a-42ca-b9fa-943e7888594c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.241102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6b6db7413f41d2bdc092b5d1c285b2', 'request_id': 'req-0a657054-a35a-42ca-b9fa-943e7888594c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.253385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6b6db7413f41d2bdc092b5d1c285b2', 'request_id': 'req-0a657054-a35a-42ca-b9fa-943e7888594c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.448554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] The token cache is being invalidate because project 23f7ceb53f794eb39f7cb346bdebcfdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:27.455523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.455523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.455523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 842/1674] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:26 2022] DELETE /identity/v3/projects/23f7ceb53f794eb39f7cb346bdebcfdf => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:27.457212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.458620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:27.504581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] The token cache is being invalidate because project fb6b6db7413f41d2bdc092b5d1c285b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:27.506140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.506140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0a657054-a35a-42ca-b9fa-943e7888594c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:27.508329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 833/1675] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:26 2022] DELETE /identity/v3/projects/fb6b6db7413f41d2bdc092b5d1c285b2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:27.510480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.511024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.511971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.518886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.649154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.650583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4522b-8f5f-401b-9c86-d2aecebfd82e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.670218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] MFA Rules not processed for user `0ace2374550e40ce812407406c262ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:27.672884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:27.817913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.818800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.819247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.819663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.821096 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.825841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.841146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 834/1676] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:27.842394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.863092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:27.890842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 843/1677] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:27.890946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18d93b-36ff-4d70-b49c-ac12c67e95a5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.891034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.892513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:27.899793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.900724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.901207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.902218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] PATH_INFO: `/v3/users/abdfc175c6714dd6929a165693c8c07e` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.904438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdfc175c6714dd6929a165693c8c07e)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.906867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.921410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8583a-cd73-4fb5-ba60-ae55db369d04 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.936543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.936543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.936543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.936543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] PATH_INFO: `/v3/users/b7c209a2a4dc481ab14d587cbcaea91e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.946373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c209a2a4dc481ab14d587cbcaea91e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.948621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.150481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdfc175c6714dd6929a165693c8c07e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.261775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c209a2a4dc481ab14d587cbcaea91e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.267330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdfc175c6714dd6929a165693c8c07e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.280611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdfc175c6714dd6929a165693c8c07e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.282059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdfc175c6714dd6929a165693c8c07e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.285662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c209a2a4dc481ab14d587cbcaea91e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.310070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-0cb79b9e-8dba-4c44-b8e1-9d20cc265fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdfc175c6714dd6929a165693c8c07e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.319414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 835/1678] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:27 2022] DELETE /identity/v3/users/abdfc175c6714dd6929a165693c8c07e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:28.321727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.322387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.322790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.350441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:28.352474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:28.383075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c209a2a4dc481ab14d587cbcaea91e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.383833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c209a2a4dc481ab14d587cbcaea91e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.420574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-065efe76-7617-4d00-bf2b-c8195cf33fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c209a2a4dc481ab14d587cbcaea91e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:28.439681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 844/1679] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:27 2022] DELETE /identity/v3/users/b7c209a2a4dc481ab14d587cbcaea91e => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:28.446011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.447933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.448861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.451317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 836/1680] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:28.463224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.463224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:28.530965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] MFA Rules not processed for user `4dfd3ee79da943b189824630b5c393ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:28.532735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:28.555443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010578fd-7b5a-40af-8cac-197cea133643 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.707936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 845/1681] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:28.708869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.710328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:28.735466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.736382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.736665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.736935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.742476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:28.746116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.868289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e852a91-7c55-4e92-a087-68d04f6f235c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.878782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51088c1b-04c4-4ee0-98da-03051a8b4256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.879726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51088c1b-04c4-4ee0-98da-03051a8b4256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.880156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51088c1b-04c4-4ee0-98da-03051a8b4256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.880595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51088c1b-04c4-4ee0-98da-03051a8b4256 admin admin] PATH_INFO: `/v3/projects/dc23799963fa49c88a90ceeacf625a46/users/584102b185f3485c8552ea5c0575fa40/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.894151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.924959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51088c1b-04c4-4ee0-98da-03051a8b4256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc23799963fa49c88a90ceeacf625a46, user_id=584102b185f3485c8552ea5c0575fa40, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.929999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51088c1b-04c4-4ee0-98da-03051a8b4256 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.948444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.949993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8d89c-fc0b-4d8a-a444-c0ba651b5ae3 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.984355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 837/1682] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:28.985863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43236d-d38e-4cee-96fd-e4df2c6f5240 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.986151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43236d-d38e-4cee-96fd-e4df2c6f5240 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.986427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43236d-d38e-4cee-96fd-e4df2c6f5240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.028690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 846/1683] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:28 2022] PUT /identity/v3/projects/dc23799963fa49c88a90ceeacf625a46/users/584102b185f3485c8552ea5c0575fa40/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:29.030317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe36ec-1eaf-4197-a442-adad561b71d4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.030734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe36ec-1eaf-4197-a442-adad561b71d4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.031198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe36ec-1eaf-4197-a442-adad561b71d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.056259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-bf43236d-d38e-4cee-96fd-e4df2c6f5240 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:29.058851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bf43236d-d38e-4cee-96fd-e4df2c6f5240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.078960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1dbe36ec-1eaf-4197-a442-adad561b71d4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:29.080717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1dbe36ec-1eaf-4197-a442-adad561b71d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.206653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 847/1684] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:29.208315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 838/1685] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:29.211022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.211721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.212355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.215285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.215579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.215860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.252201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] MFA Rules not processed for user `4dfd3ee79da943b189824630b5c393ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:29.254042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.295445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] MFA Rules not processed for user `584102b185f3485c8552ea5c0575fa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:29.300412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.359526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 848/1686] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:29.360662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.389080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a07e6-3e38-4c25-90a3-38cfd130aeae None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.396108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.396931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.397241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.397517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.398267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:29.398572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.400224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.430795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 839/1687] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:29.431067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 849/1688] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:29.432167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.434839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.435276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.437998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.463222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53e24-2849-4c2d-afd3-b5748ab643b0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.478511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.479180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.479459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.479741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] PATH_INFO: `/v3/projects/eaf62c61a39a4de9b8f828db5ff749cb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.480922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf62c61a39a4de9b8f828db5ff749cb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.482891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.535503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c56001-3f61-4492-bc40-6d230f4d4d9f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.550075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.550327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.550557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.550832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.579402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.581504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.602532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 840/1689] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:29.602532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.603983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.629309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf62c61a39a4de9b8f828db5ff749cb', 'request_id': 'req-0bd7f6a3-8140-4872-96af-acebed31cfde'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:29.643764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf62c61a39a4de9b8f828db5ff749cb', 'request_id': 'req-0bd7f6a3-8140-4872-96af-acebed31cfde'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:29.654127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf62c61a39a4de9b8f828db5ff749cb', 'request_id': 'req-0bd7f6a3-8140-4872-96af-acebed31cfde'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:29.705212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] The token cache is being invalidate because project eaf62c61a39a4de9b8f828db5ff749cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:29.705962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:29.711495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:29.716678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 850/1690] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:29 2022] DELETE /identity/v3/projects/eaf62c61a39a4de9b8f828db5ff749cb => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:29.718085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.719616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:29.797250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58987fc1-55a4-4940-af26-4ae8e80c9bdf service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.879766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f6a3-8140-4872-96af-acebed31cfde admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.890605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.891379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.891760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.892129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.892868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.894939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.971390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.973188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.979060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.979060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.979060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.048342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 851/1691] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:29 2022] POST /identity/v3/projects => generated 367 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:30.049454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.055716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.057252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.086321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.088666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef91a9-cfd1-4cc2-b07a-bee6e019c53c service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.108348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 841/1692] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:30.110137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.110455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.110740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.137139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:30.140661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.188065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.259471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 842/1693] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:30.260284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.277642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22a386-a63b-4612-b117-92c3024a40ae admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.279825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e1483-b71c-40e4-9beb-c106fbb8001d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.283139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50575-0e71-4c39-9003-01e3149d3d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.285999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50575-0e71-4c39-9003-01e3149d3d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.285999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50575-0e71-4c39-9003-01e3149d3d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.285999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50575-0e71-4c39-9003-01e3149d3d7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.285999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50575-0e71-4c39-9003-01e3149d3d7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.286953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50575-0e71-4c39-9003-01e3149d3d7d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.294573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d344e2-717a-48cf-9b60-989f30bfdfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.295210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d344e2-717a-48cf-9b60-989f30bfdfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.295495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d344e2-717a-48cf-9b60-989f30bfdfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.296014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d344e2-717a-48cf-9b60-989f30bfdfb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.296768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d344e2-717a-48cf-9b60-989f30bfdfb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.298655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d344e2-717a-48cf-9b60-989f30bfdfb1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.432769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 843/1694] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:30.434173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.434466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.434744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.494378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:30.496285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.501895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 852/1695] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/projects => generated 357 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:30.504417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.504417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.504417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.568667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] MFA Rules not processed for user `c9577bc4c4c546ca9fc220ebd3a16346`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:30.570269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.663344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 844/1696] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:30.664909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.699847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c49bdf-cdb9-4566-8203-1494225ba24a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.708544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 853/1697] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:30.709764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.715752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.716380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.716662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.716939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.717619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.719978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.728432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebff50-cf91-43e3-ac38-7119b50ccfca None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.736874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.738040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.738583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.739069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.740594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.743606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.750345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 845/1698] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:30.751184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.771211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b3dad-3eac-40d5-b63a-86b73e912af2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.790177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.790835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.791128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.791403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.792073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:30.792380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.794429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.819421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 846/1699] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:30.820680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.824487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.901033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 854/1700] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:30.901710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.937344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccfcaf-9570-4e68-bfce-4089164337c6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.968685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.968685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.968685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.969105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.969185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.969185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.974488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.998620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 855/1701] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:31.001774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.037033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30d697-e865-4278-bf89-858e5071e848 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.050449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1d8fc-76bf-4242-baae-0a3f7aa90a87 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.053709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.058153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.059182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.059616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.060194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.061039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.069560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.073514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.076823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.077971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.078808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.102617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.107089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.126078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 847/1702] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:31.127371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.142095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44120-b3df-4ce6-a358-f4ed85cfa1fe service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.155315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b058ed-50d1-4b2c-a259-7d26101a33e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.156598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b058ed-50d1-4b2c-a259-7d26101a33e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.157196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b058ed-50d1-4b2c-a259-7d26101a33e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.158052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b058ed-50d1-4b2c-a259-7d26101a33e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.159281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b058ed-50d1-4b2c-a259-7d26101a33e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.162501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b058ed-50d1-4b2c-a259-7d26101a33e8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.190166 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 856/1703] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:31 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:31.196268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.218129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 848/1704] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:31.220515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5896eb-6afe-420b-90f6-9e5f12468835 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.221205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5896eb-6afe-420b-90f6-9e5f12468835 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.221648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5896eb-6afe-420b-90f6-9e5f12468835 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.223821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 849/1705] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 10 13:25:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:31.226769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.227231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.228041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.306967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] MFA Rules not processed for user `a46db4eba75f4c49a394bffe4f220fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:31.309290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:31.388974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988551b4-523f-4e0c-8036-c6d1649d3ed0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.407750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c59ff1-3f36-4fc4-8678-73bfb96e1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.408493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c59ff1-3f36-4fc4-8678-73bfb96e1ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.408780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c59ff1-3f36-4fc4-8678-73bfb96e1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.409080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c59ff1-3f36-4fc4-8678-73bfb96e1ade admin admin] PATH_INFO: `/v3/projects/b6f257ef437f4ae78e0360f40d1f4b2b/users/af450ee2e4b046c99afac9cb911a0e8e/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.423356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 850/1706] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:31.424465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.439190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41b05a-3c0a-4b04-8ae0-96d7be10a79c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.446174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.453625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.455539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.455850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.456120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.458651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.462462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.513808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c59ff1-3f36-4fc4-8678-73bfb96e1ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f257ef437f4ae78e0360f40d1f4b2b, user_id=af450ee2e4b046c99afac9cb911a0e8e, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.514997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c59ff1-3f36-4fc4-8678-73bfb96e1ade admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.613689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 851/1707] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:31 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:31.614545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.650781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24ba02-10b4-4a72-92af-5bcec228e896 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.661530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.664680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.667746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.668828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] PATH_INFO: `/v3/projects/8e4b38409b03404bb8d5784b88bb0a47/users/679400986900428ea43208bb27baacce/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.672309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 857/1708] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:31 2022] PUT /identity/v3/projects/b6f257ef437f4ae78e0360f40d1f4b2b/users/af450ee2e4b046c99afac9cb911a0e8e/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:31.674042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f83c65-e96d-4405-a205-304e56cf7649 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.674633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f83c65-e96d-4405-a205-304e56cf7649 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.675038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f83c65-e96d-4405-a205-304e56cf7649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.712875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4b38409b03404bb8d5784b88bb0a47, user_id=679400986900428ea43208bb27baacce, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.717027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.757294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-67f83c65-e96d-4405-a205-304e56cf7649 None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:31.760836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-67f83c65-e96d-4405-a205-304e56cf7649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:31.813330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 852/1709] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:31 2022] PUT /identity/v3/projects/8e4b38409b03404bb8d5784b88bb0a47/users/679400986900428ea43208bb27baacce/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:31.814750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.819435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:31.915210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 858/1710] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:31.916702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.917380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.917902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.955339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] MFA Rules not processed for user `a46db4eba75f4c49a394bffe4f220fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:31.955339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.019581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c482202-a810-42df-81a8-2e540861f9de admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.045721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.046446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.046728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.047005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] PATH_INFO: `/v3/projects/d63f4e449ad24501aab056be873063fb` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.050963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63f4e449ad24501aab056be873063fb)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.055367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.092744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 859/1711] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:32.096104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.114427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314e05f-b0cb-4eb9-8658-f8c4c87f75f0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.120182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e43966-6c16-4e2d-b711-606a71ba713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.120949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43966-6c16-4e2d-b711-606a71ba713e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.121335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43966-6c16-4e2d-b711-606a71ba713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.121616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43966-6c16-4e2d-b711-606a71ba713e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.122385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43966-6c16-4e2d-b711-606a71ba713e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.123903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43966-6c16-4e2d-b711-606a71ba713e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.156573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 860/1712] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:32.158414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.158708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.158984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.251749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] MFA Rules not processed for user `af450ee2e4b046c99afac9cb911a0e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:32.258147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.278291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63f4e449ad24501aab056be873063fb', 'request_id': 'req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:32.293540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63f4e449ad24501aab056be873063fb', 'request_id': 'req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:32.302041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63f4e449ad24501aab056be873063fb', 'request_id': 'req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:32.396218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 861/1713] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:32.401342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.402092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] The token cache is being invalidate because project d63f4e449ad24501aab056be873063fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:32.402092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:32.402516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:32.405952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 853/1714] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:31 2022] DELETE /identity/v3/projects/d63f4e449ad24501aab056be873063fb => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:32.406868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.408530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.527690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a2083-8bcc-4d35-a7b6-9d789843d0ed None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.534881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.535698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.536238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.536657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.538326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c6b3f-44cb-4c00-9655-491e2aa1d367 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.539123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.541502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.551725 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.552365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.552803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.559327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.559489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 862/1715] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:32.559489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.559660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.560726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.571912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.572970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.592233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 854/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:32.593189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.598319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.619567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.628154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af265a-e4c8-4a14-9f64-9251eb8a6868 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.646050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.646440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.646789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.647046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.653165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.653165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.670360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 855/1717] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:32.671736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.676460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.707275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5066e3-f8c2-4870-bed9-35e32ecc888e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.712169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.719364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688cf0f-786b-4280-a945-b4e4aa0c2ce5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.726643 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4789c4-2e20-4036-9fa5-d975e05e91fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.727279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4789c4-2e20-4036-9fa5-d975e05e91fc admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.727560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4789c4-2e20-4036-9fa5-d975e05e91fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.727840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4789c4-2e20-4036-9fa5-d975e05e91fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.728538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4789c4-2e20-4036-9fa5-d975e05e91fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.730545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4789c4-2e20-4036-9fa5-d975e05e91fc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.734927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.735565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.736899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.736899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.740973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.847100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 856/1718] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:32 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:32.848555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.848853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.849527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.859010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.864797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.886246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 863/1719] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:32.887207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.889633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.913397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:32.918183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.920844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b8aba-2318-49a9-9745-e2f0458270de service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.079004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 857/1720] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:33.081572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.085875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.085875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.085875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] PATH_INFO: `/v3/projects/8e4b38409b03404bb8d5784b88bb0a47/users/679400986900428ea43208bb27baacce/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.086290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.093298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4b38409b03404bb8d5784b88bb0a47, user_id=679400986900428ea43208bb27baacce, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.096945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.117874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 864/1721] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:32 2022] PUT /identity/v3/projects/8e4b38409b03404bb8d5784b88bb0a47/users/679400986900428ea43208bb27baacce/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:33.125480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.129073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7236f-ff60-4f02-abb6-277f102b9a95 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.288054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fbe50-f90f-4fe9-b719-a5bd8ed328c4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.300751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.304580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f6e33-54b6-458a-8588-4accd7812881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.308162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f6e33-54b6-458a-8588-4accd7812881 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.308162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f6e33-54b6-458a-8588-4accd7812881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.308162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f6e33-54b6-458a-8588-4accd7812881 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.308162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f6e33-54b6-458a-8588-4accd7812881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.311619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f6e33-54b6-458a-8588-4accd7812881 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.331260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.420298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.421160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.421438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.421716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] PATH_INFO: `/v3/projects/ee5d790c63494a51bd42c74d1949254f/users/2f072c6193b84db3af4a9aaeec8636af/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.448278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 865/1722] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:33 2022] POST /identity/v3/users => generated 289 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:33.451433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41471bab-5b57-4263-99b4-f0087fe58bff None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.451824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41471bab-5b57-4263-99b4-f0087fe58bff None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.453063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41471bab-5b57-4263-99b4-f0087fe58bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.481669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5d790c63494a51bd42c74d1949254f, user_id=2f072c6193b84db3af4a9aaeec8636af, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.484402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.521478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-41471bab-5b57-4263-99b4-f0087fe58bff None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:33.528539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-41471bab-5b57-4263-99b4-f0087fe58bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:33.555811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 866/1723] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:33.555811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.555811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.555811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.613166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 858/1724] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:33 2022] PUT /identity/v3/projects/ee5d790c63494a51bd42c74d1949254f/users/2f072c6193b84db3af4a9aaeec8636af/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:33.614252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.641032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] MFA Rules not processed for user `679400986900428ea43208bb27baacce`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:33.642363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:33.778799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237850-a0d2-422a-93f8-dd58cc141f8e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.780344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 867/1725] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:33.781544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.805355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd59720-934f-4138-8679-c3fd47bbae32 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.807101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682545d-4519-4870-ad52-b3ed696fa5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.807688 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682545d-4519-4870-ad52-b3ed696fa5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.807973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682545d-4519-4870-ad52-b3ed696fa5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.808263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682545d-4519-4870-ad52-b3ed696fa5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.811403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682545d-4519-4870-ad52-b3ed696fa5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.812684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.815518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.816039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.816518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.817358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682545d-4519-4870-ad52-b3ed696fa5ad admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.817884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:33.818399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.822211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.836243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 859/1726] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:33.841927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.842290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.842581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.847756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 868/1727] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:33.849330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.853210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:33.958209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] MFA Rules not processed for user `2f072c6193b84db3af4a9aaeec8636af`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:33.959446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:34.022619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303e935-430d-46ed-aab9-5aa5539bb478 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.028463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.029121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.029402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.029677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.041302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.048158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.051532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.070670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 860/1728] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:34.075602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.095856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 869/1729] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:34.095856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.118603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.120881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33a40e-e731-4df0-bcd2-4bd8fd8a1d35 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.124285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e2355-fa1d-4e57-a4bb-78beb1cbc28a service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.131444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db758-91a4-4a00-b3bb-65113b01840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.133109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db758-91a4-4a00-b3bb-65113b01840f admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.133271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db758-91a4-4a00-b3bb-65113b01840f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.134049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db758-91a4-4a00-b3bb-65113b01840f admin admin] PATH_INFO: `/v3/projects/26b857340675408992aa9ecf2aaba06a/users/a08dc02bae274d6fbde66b177b5934d3/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.136259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05036e-a5ac-4c2f-8618-5d5bc073d98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.136826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05036e-a5ac-4c2f-8618-5d5bc073d98a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.137134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05036e-a5ac-4c2f-8618-5d5bc073d98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.137406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05036e-a5ac-4c2f-8618-5d5bc073d98a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.138298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05036e-a5ac-4c2f-8618-5d5bc073d98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.140236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05036e-a5ac-4c2f-8618-5d5bc073d98a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.180916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db758-91a4-4a00-b3bb-65113b01840f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b857340675408992aa9ecf2aaba06a, user_id=a08dc02bae274d6fbde66b177b5934d3, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.183948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db758-91a4-4a00-b3bb-65113b01840f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.268681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 870/1730] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:34 2022] POST /identity/v3/projects => generated 393 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:34.270569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b4f13-bfc5-4eda-ac2b-3b98da47a4c6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.270963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b4f13-bfc5-4eda-ac2b-3b98da47a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.271306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b4f13-bfc5-4eda-ac2b-3b98da47a4c6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.273349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 871/1731] 10.222.0.40 () {54 vars in 961 bytes} [Wed Aug 10 13:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:34.274836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b4f13-bfc5-4eda-ac2b-3b98da47a4c6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.293052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 861/1732] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:34 2022] PUT /identity/v3/projects/26b857340675408992aa9ecf2aaba06a/users/a08dc02bae274d6fbde66b177b5934d3/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:34.294460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.294755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.295037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.331237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] MFA Rules not processed for user `4dfd3ee79da943b189824630b5c393ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:34.332417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:34.392341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b4f13-bfc5-4eda-ac2b-3b98da47a4c6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.451522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 862/1733] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:34.453249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.470602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.560588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1d544-5c04-4010-8124-ad15b1a9cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.561712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1d544-5c04-4010-8124-ad15b1a9cbfd admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.566467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1d544-5c04-4010-8124-ad15b1a9cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.566467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1d544-5c04-4010-8124-ad15b1a9cbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.566467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1d544-5c04-4010-8124-ad15b1a9cbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.566467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1d544-5c04-4010-8124-ad15b1a9cbfd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.618938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f56a4b-ffdb-419e-b2e0-e8db79b1be2d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.630142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.630930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.631306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.631748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.633278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.636299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.690707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 872/1734] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:34 2022] POST /identity/v3/projects => generated 363 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:34.690707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.690707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.691826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.743531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:34.746457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:34.768555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 863/1735] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:34 2022] POST /identity/v3/users => generated 297 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:34.772593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.799225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25acad1-585c-49c9-bf9b-6de2cd63e983 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.912153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 873/1736] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Aug 10 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:34.913129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.967069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3276-3f68-4572-9632-ce1d26460806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.967709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3276-3f68-4572-9632-ce1d26460806 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.967993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3276-3f68-4572-9632-ce1d26460806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.969876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3276-3f68-4572-9632-ce1d26460806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.970976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3276-3f68-4572-9632-ce1d26460806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.975467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3276-3f68-4572-9632-ce1d26460806 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.004652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 864/1737] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:35.010590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd743582-da23-492e-86d6-c6ae4ae821cb None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.010590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd743582-da23-492e-86d6-c6ae4ae821cb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.010590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd743582-da23-492e-86d6-c6ae4ae821cb None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.010590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 865/1738] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 10 13:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:35.010590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd743582-da23-492e-86d6-c6ae4ae821cb None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.025303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4645d4-782b-499f-91bc-253d973395a7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.033376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.034020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.034306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.034765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.035563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd743582-da23-492e-86d6-c6ae4ae821cb None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.037413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:35.042558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.043229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.043787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.044070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.045035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.047012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.052427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.125267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.126701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.137944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 874/1739] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:35.144091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.147433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 866/1740] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:35 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:35.150837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.152447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3ce2d-2d52-42ca-b315-32179c1abd26 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.161227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.162114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.162577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.162869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.163969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.165394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.181526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 875/1741] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:35.182991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.200474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9130-bdfa-48a0-bcc7-a5ce7219723d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.201250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6421-384c-48a8-a794-ac7d2787f2aa admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.218610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.226445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a37969-4ff6-43b3-9a39-e874049ca569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.226445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37969-4ff6-43b3-9a39-e874049ca569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.226445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37969-4ff6-43b3-9a39-e874049ca569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.226445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37969-4ff6-43b3-9a39-e874049ca569 admin admin] PATH_INFO: `/v3/projects/26b857340675408992aa9ecf2aaba06a/users/a08dc02bae274d6fbde66b177b5934d3/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.227025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.227025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.227025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.227349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a37969-4ff6-43b3-9a39-e874049ca569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b857340675408992aa9ecf2aaba06a, user_id=a08dc02bae274d6fbde66b177b5934d3, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.230135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:35.232793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a37969-4ff6-43b3-9a39-e874049ca569 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.255816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 876/1742] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:35 2022] PUT /identity/v3/projects/26b857340675408992aa9ecf2aaba06a/users/a08dc02bae274d6fbde66b177b5934d3/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:35.257389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.257683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.257958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.273706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.276633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.311881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 867/1743] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:35.312418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:35.312773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.313694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:35.316898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.334031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0268d-a0d5-4618-be39-c76cff4e87e7 service placement] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.421276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 877/1744] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:35.422751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.456504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9c039-28f9-4b9f-9497-5489c55c1a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.457239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9c039-28f9-4b9f-9497-5489c55c1a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.457590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9c039-28f9-4b9f-9497-5489c55c1a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.457932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9c039-28f9-4b9f-9497-5489c55c1a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.458820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9c039-28f9-4b9f-9497-5489c55c1a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.463617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9c039-28f9-4b9f-9497-5489c55c1a9b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.486026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 868/1745] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:35.489332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.489708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.490058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.570270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] MFA Rules not processed for user `a08dc02bae274d6fbde66b177b5934d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:35.573144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:35.573522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583b0ad-04a5-4f0a-b43a-259b20e51e9f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.581805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.582447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.582732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.583009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.583998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.587250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.618576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 878/1746] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:35.626704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.631341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:35.674935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef19010-e8c6-4e31-876b-b28be65a830f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.685282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.693392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.693392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.693392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.722784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.723741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.754547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 879/1747] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:35.756828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.798418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1bdfc-2d35-47cc-baa4-ebf9d2c7cffa service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.814904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 869/1748] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:35.819501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.826098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.826098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.826098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.826415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] PATH_INFO: `/v3/projects/63751aa0b81d44a89f9f0a4feb4436d2/users/0abb073f8efc402385ff2a3ebdcd90d7/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.853898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63751aa0b81d44a89f9f0a4feb4436d2, user_id=0abb073f8efc402385ff2a3ebdcd90d7, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.856604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.858962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c2767-1140-47d5-94aa-b7152d440181 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.885842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fff3e-1f40-4053-92c3-4b0db54786fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.887367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fff3e-1f40-4053-92c3-4b0db54786fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.887660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fff3e-1f40-4053-92c3-4b0db54786fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.887946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fff3e-1f40-4053-92c3-4b0db54786fa admin admin] PATH_INFO: `/v3/projects/c1c3ee21d9f44f3595bbe99cd3f0d08c/users/3ba73331c5c149f7b2808e013b88ac65/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.921615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fff3e-1f40-4053-92c3-4b0db54786fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c3ee21d9f44f3595bbe99cd3f0d08c, user_id=3ba73331c5c149f7b2808e013b88ac65, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.924531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fff3e-1f40-4053-92c3-4b0db54786fa admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.961944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 880/1749] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:35 2022] PUT /identity/v3/projects/63751aa0b81d44a89f9f0a4feb4436d2/users/0abb073f8efc402385ff2a3ebdcd90d7/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:35.963132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.057324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 870/1750] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:35 2022] PUT /identity/v3/projects/c1c3ee21d9f44f3595bbe99cd3f0d08c/users/3ba73331c5c149f7b2808e013b88ac65/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:36.059068 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.059420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.059770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.065951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.098311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] MFA Rules not processed for user `0ace2374550e40ce812407406c262ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:36.099837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:36.114845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ff2a3-ce2a-4fa0-811d-414455ac3c2c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.188685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.208001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 871/1751] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:36.208886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.249535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.250403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.264467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.265642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.295858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24477ea-ca4c-45df-bc59-33b7e311377f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.302043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ed720-29ba-4a9d-a50f-38eabc5b884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.302799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ed720-29ba-4a9d-a50f-38eabc5b884d admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.303148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ed720-29ba-4a9d-a50f-38eabc5b884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.303425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ed720-29ba-4a9d-a50f-38eabc5b884d admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.304347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ed720-29ba-4a9d-a50f-38eabc5b884d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.306911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ed720-29ba-4a9d-a50f-38eabc5b884d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.325631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 872/1752] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:36.327753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251423b3-b366-4473-8663-a497613c9f80 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.328053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251423b3-b366-4473-8663-a497613c9f80 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.328372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251423b3-b366-4473-8663-a497613c9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.364120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-251423b3-b366-4473-8663-a497613c9f80 None None] MFA Rules not processed for user `3ba73331c5c149f7b2808e013b88ac65`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:36.365581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-251423b3-b366-4473-8663-a497613c9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:36.458504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 873/1753] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:36.459582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251423b3-b366-4473-8663-a497613c9f80 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.472587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251423b3-b366-4473-8663-a497613c9f80 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.480500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdd418-c648-43a8-9522-93be27010949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.482127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdd418-c648-43a8-9522-93be27010949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.482127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdd418-c648-43a8-9522-93be27010949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.482427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdd418-c648-43a8-9522-93be27010949 admin admin] PATH_INFO: `/v3/projects/63751aa0b81d44a89f9f0a4feb4436d2/users/0abb073f8efc402385ff2a3ebdcd90d7/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.485982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdd418-c648-43a8-9522-93be27010949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63751aa0b81d44a89f9f0a4feb4436d2, user_id=0abb073f8efc402385ff2a3ebdcd90d7, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.489297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdd418-c648-43a8-9522-93be27010949 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.500778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.502635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.513675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 874/1754] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:36 2022] PUT /identity/v3/projects/63751aa0b81d44a89f9f0a4feb4436d2/users/0abb073f8efc402385ff2a3ebdcd90d7/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:36.513675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a149-9969-428d-b74c-297b3a6a644f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.513675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a149-9969-428d-b74c-297b3a6a644f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.513675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a149-9969-428d-b74c-297b3a6a644f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.528588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 881/1755] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:36.529484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.548058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372b070-a5b1-420f-929f-4410e3016612 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.562482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-bed2a149-9969-428d-b74c-297b3a6a644f None None] MFA Rules not processed for user `c9577bc4c4c546ca9fc220ebd3a16346`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:36.565599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bed2a149-9969-428d-b74c-297b3a6a644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:36.705484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 875/1756] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:36.713610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.713610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.713610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.761670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.763092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.763372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.763644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.766137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.769194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.794466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] MFA Rules not processed for user `0abb073f8efc402385ff2a3ebdcd90d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:36.797678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:36.941665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 882/1757] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:36 2022] POST /identity/v3/projects => generated 391 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:36.942688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.950142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:36.993952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 876/1758] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:36.995591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.020014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8556c94-a043-4f16-a451-cce4a7ad6677 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.217836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a7ac4-515d-4f03-b279-81dadf5f416c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.227124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.227763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.228047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.228322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.231473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.232075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.232359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.232636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.233965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:37.235578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.239552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.338687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.340304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.358327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 883/1759] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:37.375983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 877/1760] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:36 2022] POST /identity/v3/users => generated 302 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:37.379976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.448050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0921e46-6987-4850-bd82-48c546649a4d service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.460833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.467890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.470962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.470962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.471798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.472560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.473115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.474846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.490119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289d405-97e1-4e9a-9b5b-56c8292bb6ea admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.509113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 884/1761] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:37.509749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.510087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.515864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.516168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.517498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.535779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e77bb-cf1a-44e3-8fef-fd2e3511b370 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.545793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.550237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.551478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.551956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.552613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] PATH_INFO: `/v3/projects/e13414596a4045a886b53496ce0c60bf/users/ce2bdc64c96b404fb0623225528682de/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.580687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13414596a4045a886b53496ce0c60bf, user_id=ce2bdc64c96b404fb0623225528682de, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.584728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.671250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 885/1762] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:37 2022] PUT /identity/v3/projects/e13414596a4045a886b53496ce0c60bf/users/ce2bdc64c96b404fb0623225528682de/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:37.676201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.683548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.687308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.731220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 878/1763] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:37.772238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199cdf5-1841-4d1f-b226-673bfc0cf146 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.779814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da50f4-0844-4502-94d6-814f3f3e9a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.780633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da50f4-0844-4502-94d6-814f3f3e9a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.781220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da50f4-0844-4502-94d6-814f3f3e9a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.781680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da50f4-0844-4502-94d6-814f3f3e9a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.783008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da50f4-0844-4502-94d6-814f3f3e9a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.785236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da50f4-0844-4502-94d6-814f3f3e9a46 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.804705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 886/1764] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:37.806844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.824384 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548b1e-7f5a-4583-9fbe-4fec0053c70b service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.833176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.833858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.834140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.834654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] PATH_INFO: `/v3/projects/e13414596a4045a886b53496ce0c60bf/users/ce2bdc64c96b404fb0623225528682de/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.849432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13414596a4045a886b53496ce0c60bf, user_id=ce2bdc64c96b404fb0623225528682de, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.861656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.909860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 879/1765] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:37 2022] PUT /identity/v3/projects/e13414596a4045a886b53496ce0c60bf/users/ce2bdc64c96b404fb0623225528682de/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:37.922442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.922973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.923390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.986492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] MFA Rules not processed for user `ce2bdc64c96b404fb0623225528682de`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:37.988123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:38.168958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 887/1766] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:38.394903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.439792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.469424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a3f03-9341-4832-9e7e-edf26ddfe8f7 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.474051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.474721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.475002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.475275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.579392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.584621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.586732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.597894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 880/1767] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:39.078641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.090182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d354e-9555-48eb-a721-b4a4f8da78f7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.093917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedefa8f-efc1-48d9-90d2-c162e96ad11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.094527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedefa8f-efc1-48d9-90d2-c162e96ad11f service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.094805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedefa8f-efc1-48d9-90d2-c162e96ad11f service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.098382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedefa8f-efc1-48d9-90d2-c162e96ad11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.106962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedefa8f-efc1-48d9-90d2-c162e96ad11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:39.108530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedefa8f-efc1-48d9-90d2-c162e96ad11f service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.118864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 888/1768] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:39.358210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.370238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5468051-19b6-41af-a422-2da74fceed3d service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.376133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.376947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.377334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.377614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.461858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:39.463367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.489101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 881/1769] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:39.694850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.553529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61760c6a-df57-45b8-8437-f25f10b0dc2a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.554464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61760c6a-df57-45b8-8437-f25f10b0dc2a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.554891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61760c6a-df57-45b8-8437-f25f10b0dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.657379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-61760c6a-df57-45b8-8437-f25f10b0dc2a None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:41.659786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-61760c6a-df57-45b8-8437-f25f10b0dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:41.744400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.847418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 889/1770] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:42.245814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.247299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:42.462476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765dfc1-9e50-41e2-ad85-6ea037949b18 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.470304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.471383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.471956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.472398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.500583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.503507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.517754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 882/1771] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:25:42.995231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c2e9-4670-4adb-9c18-2b35fc6daf96 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.996136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c2e9-4670-4adb-9c18-2b35fc6daf96 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.996563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c2e9-4670-4adb-9c18-2b35fc6daf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.081622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-57e7c2e9-4670-4adb-9c18-2b35fc6daf96 None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:43.085606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-57e7c2e9-4670-4adb-9c18-2b35fc6daf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:43.295071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 890/1772] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 10 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:43.406610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.427420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62924aa-eaf2-4279-90eb-e70fdb265b73 service swift] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.435252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a5ae2-d40d-4cb4-95a5-40421f3ef799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.438117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5ae2-d40d-4cb4-95a5-40421f3ef799 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.438117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5ae2-d40d-4cb4-95a5-40421f3ef799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.438117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5ae2-d40d-4cb4-95a5-40421f3ef799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.473575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a5ae2-d40d-4cb4-95a5-40421f3ef799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.475110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a5ae2-d40d-4cb4-95a5-40421f3ef799 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:43.518452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 883/1773] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:46.510060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046fbe-20c1-4ae2-a8f8-cf33ad4214b3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.510673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046fbe-20c1-4ae2-a8f8-cf33ad4214b3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.510862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046fbe-20c1-4ae2-a8f8-cf33ad4214b3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.512324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 891/1774] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:46.530821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.531383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.531383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.552943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:46.554188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:46.625657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 884/1775] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:46.637519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed046fbe-20c1-4ae2-a8f8-cf33ad4214b3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.653348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed046fbe-20c1-4ae2-a8f8-cf33ad4214b3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.659050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a36f16-3e1e-44a6-adfa-ef9d788621a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.659653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36f16-3e1e-44a6-adfa-ef9d788621a6 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.660386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36f16-3e1e-44a6-adfa-ef9d788621a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.660386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36f16-3e1e-44a6-adfa-ef9d788621a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.669452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a36f16-3e1e-44a6-adfa-ef9d788621a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.671041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a36f16-3e1e-44a6-adfa-ef9d788621a6 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.681483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 892/1776] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:49.054789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.088549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.168909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d6e2-0715-40e8-abd1-0dc2cd19c1f9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.177534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.177534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.177534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.177534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] PATH_INFO: `/v3/users/af450ee2e4b046c99afac9cb911a0e8e` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.180585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af450ee2e4b046c99afac9cb911a0e8e)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.182882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.365981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af450ee2e4b046c99afac9cb911a0e8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:49.442115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af450ee2e4b046c99afac9cb911a0e8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:49.452905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af450ee2e4b046c99afac9cb911a0e8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:49.454171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af450ee2e4b046c99afac9cb911a0e8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:49.477375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af450ee2e4b046c99afac9cb911a0e8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:49.493062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 885/1777] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:49 2022] DELETE /identity/v3/users/af450ee2e4b046c99afac9cb911a0e8e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:49.493270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.493270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.493270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.520735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:49.522335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:49.629017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.631790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:49.674988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 893/1778] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:49.688108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.690630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:49.788661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971be91-c109-400b-b2b2-7a3413df43b1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.802898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cbd96-a255-47d6-8429-8140b0a034e1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.804495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.806240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.806617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.806900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] PATH_INFO: `/v3/users/3ba73331c5c149f7b2808e013b88ac65` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.810562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba73331c5c149f7b2808e013b88ac65)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.811333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09c1fa-c4c2-4b3f-9c8a-f1549657b29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.813499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.813723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c1fa-c4c2-4b3f-9c8a-f1549657b29b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.816890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c1fa-c4c2-4b3f-9c8a-f1549657b29b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.816890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c1fa-c4c2-4b3f-9c8a-f1549657b29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.841897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c1fa-c4c2-4b3f-9c8a-f1549657b29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.845645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c1fa-c4c2-4b3f-9c8a-f1549657b29b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.856194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 894/1779] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:25:50.053405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.062229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba73331c5c149f7b2808e013b88ac65'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:50.133947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba73331c5c149f7b2808e013b88ac65'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:50.145931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba73331c5c149f7b2808e013b88ac65'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:50.145931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba73331c5c149f7b2808e013b88ac65'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:50.181964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba73331c5c149f7b2808e013b88ac65'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:50.193977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 886/1780] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:49 2022] DELETE /identity/v3/users/3ba73331c5c149f7b2808e013b88ac65 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:50.201399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.202249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.203135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.238100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:50.239774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:50.253689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.331428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 895/1781] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:50.357846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.360012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:50.565425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501d9a5-f3af-466c-ad27-77a9d41f69e5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.572496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.573281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.573563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.573897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.606905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.610209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.645397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 887/1782] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:51.831290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.833356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:51.859524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c06b0-a337-414e-89bb-d5919c86a440 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.865039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.867071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.867071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.867652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] PATH_INFO: `/v3/projects/b6f257ef437f4ae78e0360f40d1f4b2b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.875094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f257ef437f4ae78e0360f40d1f4b2b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.877529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.054175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f257ef437f4ae78e0360f40d1f4b2b', 'request_id': 'req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.068199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f257ef437f4ae78e0360f40d1f4b2b', 'request_id': 'req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.084399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f257ef437f4ae78e0360f40d1f4b2b', 'request_id': 'req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.168433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] The token cache is being invalidate because project b6f257ef437f4ae78e0360f40d1f4b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:52.168970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.172110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.173893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 896/1783] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:51 2022] DELETE /identity/v3/projects/b6f257ef437f4ae78e0360f40d1f4b2b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:52.178027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.189375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be931af4-43e1-4312-bccd-44b31f0555a8 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.269274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.270295 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.270790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.271090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] PATH_INFO: `/v3/users/a08dc02bae274d6fbde66b177b5934d3` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.272340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08dc02bae274d6fbde66b177b5934d3)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.274449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.527434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08dc02bae274d6fbde66b177b5934d3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.607839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08dc02bae274d6fbde66b177b5934d3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.622190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08dc02bae274d6fbde66b177b5934d3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.625851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08dc02bae274d6fbde66b177b5934d3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.626565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.626947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.628716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:52.651269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f10b-b9c8-4f5f-9f33-c5bdd4accaaf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.675324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08dc02bae274d6fbde66b177b5934d3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.675939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.675939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.675939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.675939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] PATH_INFO: `/v3/projects/c1c3ee21d9f44f3595bbe99cd3f0d08c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.676216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c3ee21d9f44f3595bbe99cd3f0d08c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.676216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.679005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 888/1784] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:52 2022] DELETE /identity/v3/users/a08dc02bae274d6fbde66b177b5934d3 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:52.961849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c3ee21d9f44f3595bbe99cd3f0d08c', 'request_id': 'req-90c378f9-238b-4e62-ac22-beca4db391e5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.975674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c3ee21d9f44f3595bbe99cd3f0d08c', 'request_id': 'req-90c378f9-238b-4e62-ac22-beca4db391e5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:52.983161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c3ee21d9f44f3595bbe99cd3f0d08c', 'request_id': 'req-90c378f9-238b-4e62-ac22-beca4db391e5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.022005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.024624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:53.073703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] The token cache is being invalidate because project c1c3ee21d9f44f3595bbe99cd3f0d08c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:53.075384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.077743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.082094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 897/1785] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:52 2022] DELETE /identity/v3/projects/c1c3ee21d9f44f3595bbe99cd3f0d08c => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:53.087844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.096441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:53.159721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.254418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a3b76-c8b7-4ff1-8367-e9e368d00b1b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.288515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c378f9-238b-4e62-ac22-beca4db391e5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.306013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.306678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.306963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.307243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] PATH_INFO: `/v3/users/ce2bdc64c96b404fb0623225528682de` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.308404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2bdc64c96b404fb0623225528682de)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.310391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.397146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.398794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.399463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.400094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] PATH_INFO: `/v3/users/2f072c6193b84db3af4a9aaeec8636af` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.402456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f072c6193b84db3af4a9aaeec8636af)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.405137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.488501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2bdc64c96b404fb0623225528682de'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.506223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2bdc64c96b404fb0623225528682de'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.625215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2bdc64c96b404fb0623225528682de'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.626334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2bdc64c96b404fb0623225528682de'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.670018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19e02547-bdcc-44ff-8c77-6d2932debb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2bdc64c96b404fb0623225528682de'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.687896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f072c6193b84db3af4a9aaeec8636af'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.690780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 898/1786] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:53 2022] DELETE /identity/v3/users/ce2bdc64c96b404fb0623225528682de => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:53.697882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9f35-ee60-43b8-9931-6b0b601ade84 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.699282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9f35-ee60-43b8-9931-6b0b601ade84 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.700156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9f35-ee60-43b8-9931-6b0b601ade84 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.706245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 899/1787] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:25:53 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:53.769770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.770080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.770649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.773487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f072c6193b84db3af4a9aaeec8636af'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.781842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f072c6193b84db3af4a9aaeec8636af'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.783745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f072c6193b84db3af4a9aaeec8636af'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.804859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:53.807950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:53.827093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-298c8a2c-144d-4870-a306-397008c55e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f072c6193b84db3af4a9aaeec8636af'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:53.837048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 889/1788] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:53 2022] DELETE /identity/v3/users/2f072c6193b84db3af4a9aaeec8636af => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:53.840824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f953d-7657-4063-b264-845abce80653 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.841406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f953d-7657-4063-b264-845abce80653 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.841843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f953d-7657-4063-b264-845abce80653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.899474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-300f953d-7657-4063-b264-845abce80653 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:53.905730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-300f953d-7657-4063-b264-845abce80653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:54.050519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 900/1789] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:25:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:54.141683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 890/1790] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:54.207627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.209467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:54.272932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.280281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f953d-7657-4063-b264-845abce80653 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.285317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-300f953d-7657-4063-b264-845abce80653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:54.359385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e6b44-43ce-4172-ba3e-ff751142773c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.474945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f953d-7657-4063-b264-845abce80653 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.485913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.486936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.487480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.487917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.502216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7069541-4a2d-4733-8646-3c9e50bd6697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.505521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7069541-4a2d-4733-8646-3c9e50bd6697 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.507717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7069541-4a2d-4733-8646-3c9e50bd6697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.508951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7069541-4a2d-4733-8646-3c9e50bd6697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.512890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.518060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.551386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7069541-4a2d-4733-8646-3c9e50bd6697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.556793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7069541-4a2d-4733-8646-3c9e50bd6697 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.579635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 891/1791] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:54.585796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.594114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 901/1792] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:55.175777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.179082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:55.321986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869855-d3cd-4b15-9e2e-e42ab703d087 service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.349998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.350715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.350997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.351274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] PATH_INFO: `/v3/projects/26b857340675408992aa9ecf2aaba06a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.356193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b857340675408992aa9ecf2aaba06a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.363677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.516005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b857340675408992aa9ecf2aaba06a', 'request_id': 'req-4e345521-efa0-4724-9252-5343f39ada6e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:55.525105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b857340675408992aa9ecf2aaba06a', 'request_id': 'req-4e345521-efa0-4724-9252-5343f39ada6e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:55.537669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b857340675408992aa9ecf2aaba06a', 'request_id': 'req-4e345521-efa0-4724-9252-5343f39ada6e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:55.614938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] The token cache is being invalidate because project 26b857340675408992aa9ecf2aaba06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:55.615538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:55.617424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:55.619408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 892/1793] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:55 2022] DELETE /identity/v3/projects/26b857340675408992aa9ecf2aaba06a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:55.624413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbd4e0-4e11-49ef-89a2-a7ed91a456ba None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.624927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbd4e0-4e11-49ef-89a2-a7ed91a456ba None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.625038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbd4e0-4e11-49ef-89a2-a7ed91a456ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.665641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c4fbd4e0-4e11-49ef-89a2-a7ed91a456ba None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:55.665641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c4fbd4e0-4e11-49ef-89a2-a7ed91a456ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:55.841819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 902/1794] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:55.842475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.843694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.844005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.844209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:55.844502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.868314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345521-efa0-4724-9252-5343f39ada6e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.875311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.875973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.877176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.877176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] PATH_INFO: `/v3/projects/e13414596a4045a886b53496ce0c60bf` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.878162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13414596a4045a886b53496ce0c60bf)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.880372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.899402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:55.900910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.902833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:55.929145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 903/1795] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:55.934012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.963472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77956bb-fa3c-4c03-91ce-fbc05825fd74 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.991010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.992381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.992659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.992928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.993650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:55.994032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.995553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.030843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 904/1796] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.039011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.060744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bb398-51a2-4528-8517-603aaa7edddf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.085847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13414596a4045a886b53496ce0c60bf', 'request_id': 'req-57faa085-bf16-4ef2-8164-50e4eb4495a7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.092842 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13414596a4045a886b53496ce0c60bf', 'request_id': 'req-57faa085-bf16-4ef2-8164-50e4eb4495a7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.104002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13414596a4045a886b53496ce0c60bf', 'request_id': 'req-57faa085-bf16-4ef2-8164-50e4eb4495a7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.187334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.188958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.189091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.189347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.189986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.193030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.217346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.419553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] The token cache is being invalidate because project e13414596a4045a886b53496ce0c60bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:56.420174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.420486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-57faa085-bf16-4ef2-8164-50e4eb4495a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.422320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 893/1797] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:55 2022] DELETE /identity/v3/projects/e13414596a4045a886b53496ce0c60bf => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:56.427069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44457ca-d730-4c64-9c8e-a893ee9972b0 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.427422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44457ca-d730-4c64-9c8e-a893ee9972b0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.427771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44457ca-d730-4c64-9c8e-a893ee9972b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.449699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c44457ca-d730-4c64-9c8e-a893ee9972b0 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:56.450920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c44457ca-d730-4c64-9c8e-a893ee9972b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.462941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 894/1798] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:56.474569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.474569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.474569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.474828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 905/1799] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:56 2022] POST /identity/v3/projects => generated 379 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:56.477047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.479025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.496032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:56.497373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.580386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31244406-6967-47c1-a20a-8406583b00e7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.585735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 895/1800] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:56.586657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.590144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.590522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.591837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.592312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.592589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.593283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.595620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.603692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250b5a7-9a1b-43aa-9240-37cf894f0158 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.610774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.611438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.611721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.611999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] PATH_INFO: `/v3/projects/ee5d790c63494a51bd42c74d1949254f` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.613225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5d790c63494a51bd42c74d1949254f)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.615197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.687775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 906/1801] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:56 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:56.687775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.698428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f978d52-325d-4616-9de2-eb74e009bc2d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.704254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.704948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.705350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.705767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.706466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.706989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.708788 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.718825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5d790c63494a51bd42c74d1949254f', 'request_id': 'req-c972caf9-6d3c-4579-8a0e-236f1d056e3d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.727037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5d790c63494a51bd42c74d1949254f', 'request_id': 'req-c972caf9-6d3c-4579-8a0e-236f1d056e3d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.731568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 907/1802] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.731568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.744001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5d790c63494a51bd42c74d1949254f', 'request_id': 'req-c972caf9-6d3c-4579-8a0e-236f1d056e3d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.745215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1f204-2d01-4ec5-9b41-1806386be957 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.801860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] The token cache is being invalidate because project ee5d790c63494a51bd42c74d1949254f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:56.802261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.803090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:56.808630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 896/1803] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:56 2022] DELETE /identity/v3/projects/ee5d790c63494a51bd42c74d1949254f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:56.808630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.808630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.810200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.811919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.812202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.812544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.814859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.814859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.823563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972caf9-6d3c-4579-8a0e-236f1d056e3d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.826297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 908/1804] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:56.827148 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.828789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.831520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.832175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.832475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.832749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.833404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.835583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.841297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db967f5-0c81-443a-b5d8-15dfc7114e54 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.846510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.847266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.847582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.847857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] PATH_INFO: `/v3/users/0abb073f8efc402385ff2a3ebdcd90d7` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.849225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0abb073f8efc402385ff2a3ebdcd90d7)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.851249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.851388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.905412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 897/1805] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:56 2022] POST /identity/v3/projects => generated 401 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:56.905412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.916481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:57.040851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f96ac-95e2-464e-93f9-8d24c8826e57 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.047283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.047283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.047283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.047283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.047615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.049143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.061067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 898/1806] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:57.061429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.072902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be8a65-2357-4bba-9d83-3c54ec58609e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.078231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.078817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.079099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.079372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.079996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.080793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0abb073f8efc402385ff2a3ebdcd90d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:57.082467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.090446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0abb073f8efc402385ff2a3ebdcd90d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:57.169056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 899/1807] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:57 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:57.170282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.173360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0abb073f8efc402385ff2a3ebdcd90d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:57.174063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0abb073f8efc402385ff2a3ebdcd90d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:57.175999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.180844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1b21-d8a1-4243-a3ea-94966991f59d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.188318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6f618-4713-491b-8f8c-8bb513aada5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.188318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6f618-4713-491b-8f8c-8bb513aada5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.188318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6f618-4713-491b-8f8c-8bb513aada5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.188318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6f618-4713-491b-8f8c-8bb513aada5d admin admin] PATH_INFO: `/v3/projects/9c4fbea10bdd41d2a75bbce881aefd65/users/a8289401542240b0803cb54b147cf56f/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.190195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0abb073f8efc402385ff2a3ebdcd90d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:57.197161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 909/1808] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:56 2022] DELETE /identity/v3/users/0abb073f8efc402385ff2a3ebdcd90d7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:57.203635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.205945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:57.221243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6f618-4713-491b-8f8c-8bb513aada5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4fbea10bdd41d2a75bbce881aefd65, user_id=a8289401542240b0803cb54b147cf56f, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.224371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6f618-4713-491b-8f8c-8bb513aada5d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.296223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9d5b7-6c68-41b1-b313-c3fbf6a7d1cc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.297709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.298402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.298763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.299095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.299795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.301648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.315714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 910/1809] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:57.316772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.328967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef9816-7fd1-4798-8869-2931356ae099 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.342163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 900/1810] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:57 2022] PUT /identity/v3/projects/9c4fbea10bdd41d2a75bbce881aefd65/users/a8289401542240b0803cb54b147cf56f/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:57.344964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.345693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.346036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.346376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.347112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.349083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.355191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.355770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.356722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.364811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 911/1811] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:57.367438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.380389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327128-5adc-44d4-8bad-bf3ae82000b3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.411969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] MFA Rules not processed for user `a8289401542240b0803cb54b147cf56f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:57.413844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:57.470224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926ae0d-1ba2-46f8-a320-071a6d4de64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.471190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926ae0d-1ba2-46f8-a320-071a6d4de64b admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.471402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926ae0d-1ba2-46f8-a320-071a6d4de64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.471681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926ae0d-1ba2-46f8-a320-071a6d4de64b admin admin] PATH_INFO: `/v3/projects/c3e2fda9c8d94db8a3e6794ebf630793/users/7a1212ff4b5c4c7aa6b4a04589c1c559/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.494210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926ae0d-1ba2-46f8-a320-071a6d4de64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e2fda9c8d94db8a3e6794ebf630793, user_id=7a1212ff4b5c4c7aa6b4a04589c1c559, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.498405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926ae0d-1ba2-46f8-a320-071a6d4de64b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.520405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 901/1812] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:57.526042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.532228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:57.547022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea5c79-d652-45b1-aa5c-054d5a21d7d6 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.552077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.555593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.555593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.555593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.555593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.556458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.581072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 912/1813] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:57 2022] PUT /identity/v3/projects/c3e2fda9c8d94db8a3e6794ebf630793/users/7a1212ff4b5c4c7aa6b4a04589c1c559/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:57.597513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.597513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.597513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.640634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] MFA Rules not processed for user `7a1212ff4b5c4c7aa6b4a04589c1c559`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:57.641895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:57.669779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 902/1814] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:57 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:57.670231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.704071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.704772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.734766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 913/1815] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:25:57.740608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.751239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70974c-9b5b-4c9d-a9cc-7e41d0822300 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.759678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.760345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.760626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.761265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.761899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.766060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.842610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcdb40-d9fb-45a1-82f4-3286b75bde93 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.847467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.848081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.848360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.848633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.849292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.852041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.860217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 903/1816] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:57 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:57.866740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.877462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d568dd-d25c-4ee8-a9e2-d9c2f01330c5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.883812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.884414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.884702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.884981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.885723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.887674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.899358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 904/1817] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:25:57.901959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.904393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:57.919793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2696be-06d2-450c-a06d-71097f4ec8ac admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.924516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.925178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.925476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.925867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] PATH_INFO: `/v3/projects/63751aa0b81d44a89f9f0a4feb4436d2` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.926997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63751aa0b81d44a89f9f0a4feb4436d2)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.928954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.938032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 914/1818] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:57 2022] POST /identity/v3/projects => generated 401 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:57.938032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.000712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fa555-e6a7-4a50-a22d-f1ce5e17d93f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.005383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.006042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.006324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.007364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] PATH_INFO: `/v3/projects/4d51174af5fd4f069e42f35deb6ffead/users/4b561818236e4c07bb3d16211e2bbb53/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.021793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d51174af5fd4f069e42f35deb6ffead, user_id=4b561818236e4c07bb3d16211e2bbb53, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.025277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.032556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63751aa0b81d44a89f9f0a4feb4436d2', 'request_id': 'req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:58.038579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63751aa0b81d44a89f9f0a4feb4436d2', 'request_id': 'req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:58.045358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63751aa0b81d44a89f9f0a4feb4436d2', 'request_id': 'req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:58.089485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 915/1819] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:57 2022] PUT /identity/v3/projects/4d51174af5fd4f069e42f35deb6ffead/users/4b561818236e4c07bb3d16211e2bbb53/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:58.090309 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.127276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] The token cache is being invalidate because project 63751aa0b81d44a89f9f0a4feb4436d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:25:58.127640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:58.128221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:25:58.129765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 905/1820] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:25:57 2022] DELETE /identity/v3/projects/63751aa0b81d44a89f9f0a4feb4436d2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:58.130573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.131888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.139970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:58.154989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bc601-d852-422d-96bd-b99572dd1662 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.159962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7abb3-8c62-440f-9d90-17835d373b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.160627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7abb3-8c62-440f-9d90-17835d373b72 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.161728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7abb3-8c62-440f-9d90-17835d373b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.161905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7abb3-8c62-440f-9d90-17835d373b72 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.164389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7abb3-8c62-440f-9d90-17835d373b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.164659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7abb3-8c62-440f-9d90-17835d373b72 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.205147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddccbf-6e09-47c0-a4ce-2b3727dd2352 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.210111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24c754-de63-4ac7-8a4b-a00cae7b1897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.210724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24c754-de63-4ac7-8a4b-a00cae7b1897 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.211005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24c754-de63-4ac7-8a4b-a00cae7b1897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.211280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24c754-de63-4ac7-8a4b-a00cae7b1897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.212578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24c754-de63-4ac7-8a4b-a00cae7b1897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.214304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24c754-de63-4ac7-8a4b-a00cae7b1897 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.224560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 906/1821] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:58.225824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.226119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.226397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.247455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:58.249163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.257667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 907/1822] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:58.265914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.268969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 916/1823] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/users => generated 307 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:58.268969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.268969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.268969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.276489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252410c-0792-4fa9-8f5d-f798f731df2d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.281204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.281865 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.282145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.282419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] PATH_INFO: `/v3/projects/4d51174af5fd4f069e42f35deb6ffead/users/4b561818236e4c07bb3d16211e2bbb53/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.284970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d51174af5fd4f069e42f35deb6ffead, user_id=4b561818236e4c07bb3d16211e2bbb53, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.287429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.289626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:58.290283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:58.290968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.299517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 908/1824] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:58 2022] PUT /identity/v3/projects/4d51174af5fd4f069e42f35deb6ffead/users/4b561818236e4c07bb3d16211e2bbb53/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:58.300478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.300908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 917/1825] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:58.302361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.314734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.324906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8573d-84dd-418c-9c1d-f8d65616e105 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.372379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a192-4f94-4bff-b56d-b5f5a4f1e800 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.377203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520ad9a-f17d-40a0-8f3d-0fd47950009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.377862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520ad9a-f17d-40a0-8f3d-0fd47950009d admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.378146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520ad9a-f17d-40a0-8f3d-0fd47950009d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.378424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520ad9a-f17d-40a0-8f3d-0fd47950009d admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.379077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520ad9a-f17d-40a0-8f3d-0fd47950009d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.380691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520ad9a-f17d-40a0-8f3d-0fd47950009d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.384408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.387754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.387754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.387754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.387754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:58.387754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.388339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.391022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 909/1826] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:58.394518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.394669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.394830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.401401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 918/1827] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:58.402204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.412044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ae453-f6b6-4edd-88dd-75d153bfc10a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.416791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.417530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.417806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.418077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] PATH_INFO: `/v3/projects/264cb9191d9e49d8b1781e943ad9c137/users/e9ebefbe68b64ddd935a19fa2a3fef28/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.426916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] MFA Rules not processed for user `4b561818236e4c07bb3d16211e2bbb53`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:58.428158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.433143 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=264cb9191d9e49d8b1781e943ad9c137, user_id=e9ebefbe68b64ddd935a19fa2a3fef28, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.435864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.498708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 910/1828] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:58.499615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.510089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3b592-614e-4781-9775-39d8770fa525 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.516255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.516255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.516255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.516255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.516801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.518443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.539111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 919/1829] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:58 2022] PUT /identity/v3/projects/264cb9191d9e49d8b1781e943ad9c137/users/e9ebefbe68b64ddd935a19fa2a3fef28/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:58.544930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.544930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.591149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 911/1830] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:58.591149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.619667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dff656-2f68-4ba0-88f0-15a2c71024c3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.665476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:58.675277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.675949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.676220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.676498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.677242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2bc7-8e69-4a56-a172-2e931d7fbfdb admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.682367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.682973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.683248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.683516 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.684334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.685799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:58.686013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.696920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 912/1831] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:58.698202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.709277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8624902-9465-4dc7-8974-4d5c8990889b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.716008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.716739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.717119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.717397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.718063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.720491 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.743921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.745406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.755322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 920/1832] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:58.756118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.769056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284309b-cc47-46ed-8ef3-65d1c669dcc9 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.773804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca759-e9f7-44fb-8c0c-ddf430534e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.774502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca759-e9f7-44fb-8c0c-ddf430534e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.774784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca759-e9f7-44fb-8c0c-ddf430534e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.775155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca759-e9f7-44fb-8c0c-ddf430534e84 admin admin] PATH_INFO: `/v3/projects/264cb9191d9e49d8b1781e943ad9c137/users/e9ebefbe68b64ddd935a19fa2a3fef28/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.777156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca759-e9f7-44fb-8c0c-ddf430534e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=264cb9191d9e49d8b1781e943ad9c137, user_id=e9ebefbe68b64ddd935a19fa2a3fef28, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.779818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca759-e9f7-44fb-8c0c-ddf430534e84 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.791533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 921/1833] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:58 2022] PUT /identity/v3/projects/264cb9191d9e49d8b1781e943ad9c137/users/e9ebefbe68b64ddd935a19fa2a3fef28/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:58.801711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.802252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.802691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.810412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 913/1834] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:58.818079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.829184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50365a9-fc0d-4089-bbff-fe80f8de4644 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.842704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] MFA Rules not processed for user `e9ebefbe68b64ddd935a19fa2a3fef28`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:58.844508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.913786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.915376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.915944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.916279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.927272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.927272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.950802 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 914/1835] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:58.953720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 922/1836] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:58.956706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.975803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.976034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a40f-86a5-4887-a763-c9e1553591bf admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.982197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.982950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.983271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.983544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.984200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.985850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.996654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 915/1837] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:59.002112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.014663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5f46-ba54-4bf4-91e3-04a73330a651 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.019517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.020207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.020453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.020747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] PATH_INFO: `/v3/projects/91e70510e4fa4d7391e52304c02ff20b/users/f1b26a2a2edd499d9634c3dacec9077f/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.041655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e70510e4fa4d7391e52304c02ff20b, user_id=f1b26a2a2edd499d9634c3dacec9077f, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.044896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.052075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb603c-2341-4ad5-9f7d-21a34487b96c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.056747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033986f5-3506-4c65-aae8-77818f910398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.057606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033986f5-3506-4c65-aae8-77818f910398 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.063223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033986f5-3506-4c65-aae8-77818f910398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.063223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033986f5-3506-4c65-aae8-77818f910398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.072915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033986f5-3506-4c65-aae8-77818f910398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.074434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033986f5-3506-4c65-aae8-77818f910398 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.085617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 923/1838] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:25:59.116704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 916/1839] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:59 2022] PUT /identity/v3/projects/91e70510e4fa4d7391e52304c02ff20b/users/f1b26a2a2edd499d9634c3dacec9077f/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:59.127653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.128004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.128346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.162536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] MFA Rules not processed for user `f1b26a2a2edd499d9634c3dacec9077f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:59.163686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:59.173773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.239770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 924/1840] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:25:59.244496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.305296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.308877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:59.346141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6b1e-57c6-4436-90b4-6ad50e489aad admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.362089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.362089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.362089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.362089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.372644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.374731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.413317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac377c-3c28-4cfe-a275-c86a823c7526 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.417925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.418614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.418958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.419239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.421065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:59.490120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.491856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.494128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 917/1841] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:25:59 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:59.497756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.505374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 925/1842] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:25:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:25:59.543020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.552011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.553736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:59.575122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23e202-569f-4462-969f-1fc1aa315c45 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.666308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca4a6-a7d7-4f35-88a1-2e0adc7c1df3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.686831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.687179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.688299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.688299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.688299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.695481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.762841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.764486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.764772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.765369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] PATH_INFO: `/v3/users/584102b185f3485c8552ea5c0575fa40` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.766754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=584102b185f3485c8552ea5c0575fa40)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.768871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.789071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 918/1843] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:59 2022] POST /identity/v3/users => generated 300 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:59.793384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.795686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.815255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6038-46b2-4855-8ede-78ece85a7b96 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.830920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.833119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.833623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.834339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.835215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.837867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.859414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 919/1844] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:25:59.860599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.871067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859de95d-9c6e-4013-af57-aea0eb11c989 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.875763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.883468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.883468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.883468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] PATH_INFO: `/v3/projects/3dba6764ecc0480fb4f4e1f2c533e2a8/users/18f18f3949894b4f87e99fd8057b2a21/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.895053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dba6764ecc0480fb4f4e1f2c533e2a8, user_id=18f18f3949894b4f87e99fd8057b2a21, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.897745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.008961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 920/1845] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:25:59 2022] PUT /identity/v3/projects/3dba6764ecc0480fb4f4e1f2c533e2a8/users/18f18f3949894b4f87e99fd8057b2a21/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:00.011953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.082872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584102b185f3485c8552ea5c0575fa40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.096762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584102b185f3485c8552ea5c0575fa40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.123143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f156b-7583-4fc6-8df9-d3f14d807424 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.128324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.129173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.129455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.129729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] PATH_INFO: `/v3/users/a8289401542240b0803cb54b147cf56f` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.133920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8289401542240b0803cb54b147cf56f)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.133920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.173584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584102b185f3485c8552ea5c0575fa40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.174212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584102b185f3485c8552ea5c0575fa40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.189639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584102b185f3485c8552ea5c0575fa40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.198578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 926/1846] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:25:59 2022] DELETE /identity/v3/users/584102b185f3485c8552ea5c0575fa40 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:00.198578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.200144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.275593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0797d5f-abdb-44c1-b657-279bb6384387 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.280227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c16a0c-6b16-4085-a91b-401477426414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.285479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c16a0c-6b16-4085-a91b-401477426414 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.285479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c16a0c-6b16-4085-a91b-401477426414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.285479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c16a0c-6b16-4085-a91b-401477426414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.285479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c16a0c-6b16-4085-a91b-401477426414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.285479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c16a0c-6b16-4085-a91b-401477426414 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.294612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 927/1847] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:00.295771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.296063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.296343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.314962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:00.316162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.323821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 928/1848] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:00.324553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.334496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90eefd2-81f0-4789-bd9c-f0cd3ea96717 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.351424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.351424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.351424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.351424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] PATH_INFO: `/v3/projects/3dba6764ecc0480fb4f4e1f2c533e2a8/users/18f18f3949894b4f87e99fd8057b2a21/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.352850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dba6764ecc0480fb4f4e1f2c533e2a8, user_id=18f18f3949894b4f87e99fd8057b2a21, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.352850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.360632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8289401542240b0803cb54b147cf56f'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.365098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 929/1849] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:00 2022] PUT /identity/v3/projects/3dba6764ecc0480fb4f4e1f2c533e2a8/users/18f18f3949894b4f87e99fd8057b2a21/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:00.370963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.370963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.433191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb9953-0651-4552-86e6-b321adbd8cdc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.436854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4f9c9-dc3f-4ff5-b501-108bdfd1df08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.437500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4f9c9-dc3f-4ff5-b501-108bdfd1df08 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.437782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4f9c9-dc3f-4ff5-b501-108bdfd1df08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.438051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4f9c9-dc3f-4ff5-b501-108bdfd1df08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.451060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.477262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8289401542240b0803cb54b147cf56f'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.486690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8289401542240b0803cb54b147cf56f'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.487417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8289401542240b0803cb54b147cf56f'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.498475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.503748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4f9c9-dc3f-4ff5-b501-108bdfd1df08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.503888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e7ca267c-eec5-404e-a4ec-463ab1ee05d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8289401542240b0803cb54b147cf56f'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:00.505221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4f9c9-dc3f-4ff5-b501-108bdfd1df08 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.510748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 921/1850] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:00 2022] DELETE /identity/v3/users/a8289401542240b0803cb54b147cf56f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:00.513756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.513756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.513756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.520018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 930/1851] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:00.521424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.521712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.521981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.542214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] MFA Rules not processed for user `18f18f3949894b4f87e99fd8057b2a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:00.542385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:00.543454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.543644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.613841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 931/1852] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:00.626065 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.627262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 922/1853] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:00.634950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.646325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.646325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.700632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755f171-707b-45ff-8e90-849bf893df19 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.704490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.705136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.705423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.705694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.714499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c04cd-7ae9-469b-aa51-d8f7aa505fd5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.715864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.717326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.718390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.719010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.719300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.719578 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.727741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 932/1854] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:00.732805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.732805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.745717 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 923/1855] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:00.847833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.849383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.877745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb442f8-f992-48e7-a4ab-1e6962c1cb77 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.882696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.884020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.884494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.884600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] PATH_INFO: `/v3/users/f1b26a2a2edd499d9634c3dacec9077f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.885721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b26a2a2edd499d9634c3dacec9077f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.887631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.065596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.067751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:01.094876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad223b-2885-4ad8-be9f-78d674ef7234 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.115259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.116476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.116898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.118164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] PATH_INFO: `/v3/projects/dc23799963fa49c88a90ceeacf625a46` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.120027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc23799963fa49c88a90ceeacf625a46)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.122606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.137997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b26a2a2edd499d9634c3dacec9077f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.153768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b26a2a2edd499d9634c3dacec9077f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.184787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.260262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc23799963fa49c88a90ceeacf625a46', 'request_id': 'req-e13153c5-e671-4f23-976f-b6f5bd2cfeec'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.265573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc23799963fa49c88a90ceeacf625a46', 'request_id': 'req-e13153c5-e671-4f23-976f-b6f5bd2cfeec'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.270673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b26a2a2edd499d9634c3dacec9077f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.271595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b26a2a2edd499d9634c3dacec9077f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.277558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc23799963fa49c88a90ceeacf625a46', 'request_id': 'req-e13153c5-e671-4f23-976f-b6f5bd2cfeec'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.288968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b26a2a2edd499d9634c3dacec9077f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.295361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 933/1856] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:00 2022] DELETE /identity/v3/users/f1b26a2a2edd499d9634c3dacec9077f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:01.296241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.298374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:01.366708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] The token cache is being invalidate because project dc23799963fa49c88a90ceeacf625a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:01.366998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.369705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e13153c5-e671-4f23-976f-b6f5bd2cfeec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:01.376071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 924/1857] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:01 2022] DELETE /identity/v3/projects/dc23799963fa49c88a90ceeacf625a46 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:01.377754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.378428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.378858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.390580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.392581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f80eb-c72c-4caa-ac3e-4d5e18d4ee47 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.401298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.402333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.402333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.405518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] PATH_INFO: `/v3/projects/9c4fbea10bdd41d2a75bbce881aefd65` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.407147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:01.408053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4fbea10bdd41d2a75bbce881aefd65)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.408650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:01.410210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.417931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 925/1858] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:01.423333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.428498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:01.452885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f73c2-5de2-4de8-97b1-13da66c7d18c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.458211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.458860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.464120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.464120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] PATH_INFO: `/v3/users/679400986900428ea43208bb27baacce` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.464120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=679400986900428ea43208bb27baacce)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.464120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:02.072033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4fbea10bdd41d2a75bbce881aefd65', 'request_id': 'req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.153198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4fbea10bdd41d2a75bbce881aefd65', 'request_id': 'req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.153198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4fbea10bdd41d2a75bbce881aefd65', 'request_id': 'req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.651467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] The token cache is being invalidate because project 9c4fbea10bdd41d2a75bbce881aefd65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:02.651907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.652673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.654575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 934/1859] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:01 2022] DELETE /identity/v3/projects/9c4fbea10bdd41d2a75bbce881aefd65 => generated 0 bytes in 1359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:02.655441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.657137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:02.713269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '679400986900428ea43208bb27baacce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.782117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8d0b1-026e-48cb-8cbc-3febbfb4d9a4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:02.786785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:02.787614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.788210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.788640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.790790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:02.803813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:02.814453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '679400986900428ea43208bb27baacce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.829173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '679400986900428ea43208bb27baacce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.830243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '679400986900428ea43208bb27baacce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.863676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c54613c8-6216-46ea-bc61-dade00f86b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '679400986900428ea43208bb27baacce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:02.874536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 926/1860] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:01 2022] DELETE /identity/v3/users/679400986900428ea43208bb27baacce => generated 0 bytes in 1456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:02.875897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac657e61-4d5a-46e3-83b0-e97f2e9b2bec None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.877120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac657e61-4d5a-46e3-83b0-e97f2e9b2bec None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.877120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac657e61-4d5a-46e3-83b0-e97f2e9b2bec None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.877941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 927/1861] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:02.878945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac657e61-4d5a-46e3-83b0-e97f2e9b2bec None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.880668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ac657e61-4d5a-46e3-83b0-e97f2e9b2bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:02.900169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:02.901609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf037a-12ca-4d52-8887-9d574387da9e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:02.918163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 935/1862] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:02.919766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.919973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.920302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.949411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:02.950701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:02.966671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac657e61-4d5a-46e3-83b0-e97f2e9b2bec None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:02.972521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:02.973930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.974285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.974629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] PATH_INFO: `/v3/users/4b561818236e4c07bb3d16211e2bbb53` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.976092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b561818236e4c07bb3d16211e2bbb53)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:02.979611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:03.125860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 936/1863] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:26:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:03.179359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:03.183417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:03.201147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3870d-d527-484b-ad51-36e75bf88e4b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:03.201993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b561818236e4c07bb3d16211e2bbb53'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.212644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa014b-437b-4952-8828-75997972e72c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:03.212644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa014b-437b-4952-8828-75997972e72c service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.212644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa014b-437b-4952-8828-75997972e72c service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.212644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa014b-437b-4952-8828-75997972e72c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.221802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa014b-437b-4952-8828-75997972e72c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:03.223644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa014b-437b-4952-8828-75997972e72c service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:03.235750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 937/1864] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:03.294338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b561818236e4c07bb3d16211e2bbb53'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.303841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b561818236e4c07bb3d16211e2bbb53'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.304594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b561818236e4c07bb3d16211e2bbb53'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.346625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b561818236e4c07bb3d16211e2bbb53'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.358779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 928/1865] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:02 2022] DELETE /identity/v3/users/4b561818236e4c07bb3d16211e2bbb53 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:03.626986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa014b-437b-4952-8828-75997972e72c service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:03.628441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1caa014b-437b-4952-8828-75997972e72c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:03.642524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:03.653474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa014b-437b-4952-8828-75997972e72c service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:03.656027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:03.657555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:03.772647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014086d4-ebbf-47c5-82e7-5fa8e8fd7363 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:03.778806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:03.779466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.779749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.780025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] PATH_INFO: `/v3/projects/91e70510e4fa4d7391e52304c02ff20b` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.781219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e70510e4fa4d7391e52304c02ff20b)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:03.783511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:03.814846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:03.815554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.815835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.816134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] PATH_INFO: `/v3/projects/8e4b38409b03404bb8d5784b88bb0a47` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.819138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4b38409b03404bb8d5784b88bb0a47)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:03.821589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:03.900700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e70510e4fa4d7391e52304c02ff20b', 'request_id': 'req-2e86cb24-e03e-48c5-a459-e25e4b884820'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.911044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e70510e4fa4d7391e52304c02ff20b', 'request_id': 'req-2e86cb24-e03e-48c5-a459-e25e4b884820'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.918577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e70510e4fa4d7391e52304c02ff20b', 'request_id': 'req-2e86cb24-e03e-48c5-a459-e25e4b884820'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.941765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4b38409b03404bb8d5784b88bb0a47', 'request_id': 'req-1b15f1b9-7098-40f0-adfc-eb4a86491d01'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.949442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4b38409b03404bb8d5784b88bb0a47', 'request_id': 'req-1b15f1b9-7098-40f0-adfc-eb4a86491d01'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:03.955106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4b38409b03404bb8d5784b88bb0a47', 'request_id': 'req-1b15f1b9-7098-40f0-adfc-eb4a86491d01'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.019909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] The token cache is being invalidate because project 91e70510e4fa4d7391e52304c02ff20b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:04.020111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.020839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.022685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 929/1866] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:03 2022] DELETE /identity/v3/projects/91e70510e4fa4d7391e52304c02ff20b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:04.026676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.028113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.099336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86cb24-e03e-48c5-a459-e25e4b884820 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.122707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] The token cache is being invalidate because project 8e4b38409b03404bb8d5784b88bb0a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:04.123198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.127257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b15f1b9-7098-40f0-adfc-eb4a86491d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.135291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 938/1867] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:03 2022] DELETE /identity/v3/projects/8e4b38409b03404bb8d5784b88bb0a47 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:04.140642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c47213-1b02-4de6-8c77-82c77422b8af None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.141145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c47213-1b02-4de6-8c77-82c77422b8af None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.141917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c47213-1b02-4de6-8c77-82c77422b8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.165716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.166319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.166608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.166891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] PATH_INFO: `/v3/users/18f18f3949894b4f87e99fd8057b2a21` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.168164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f18f3949894b4f87e99fd8057b2a21)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.170365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.173571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-96c47213-1b02-4de6-8c77-82c77422b8af None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:04.175130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-96c47213-1b02-4de6-8c77-82c77422b8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.185598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 939/1868] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:04.193929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.194369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.194816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.223746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:04.225048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.238800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 940/1869] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:04.241472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.259783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8d29-3edb-4a44-b967-774d9d29b293 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.270211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.271668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.271949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.272395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.273246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:04.274725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.276470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.300543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 941/1870] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:04.300543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.304856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.317877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68159de-ea9c-479a-8a4d-89406acfbc46 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.325683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.326355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.326680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.326955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] PATH_INFO: `/v3/projects/4d51174af5fd4f069e42f35deb6ffead` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.328127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d51174af5fd4f069e42f35deb6ffead)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.330191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.390896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f18f3949894b4f87e99fd8057b2a21'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.436951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d51174af5fd4f069e42f35deb6ffead', 'request_id': 'req-de95ca25-13fa-46ca-8f31-195284f32ce7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.442779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d51174af5fd4f069e42f35deb6ffead', 'request_id': 'req-de95ca25-13fa-46ca-8f31-195284f32ce7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.453235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d51174af5fd4f069e42f35deb6ffead', 'request_id': 'req-de95ca25-13fa-46ca-8f31-195284f32ce7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.493606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f18f3949894b4f87e99fd8057b2a21'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.502285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f18f3949894b4f87e99fd8057b2a21'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.503000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f18f3949894b4f87e99fd8057b2a21'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.515827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f18f3949894b4f87e99fd8057b2a21'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.521869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 930/1871] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:04 2022] DELETE /identity/v3/users/18f18f3949894b4f87e99fd8057b2a21 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:04.524435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.526008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.596165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] The token cache is being invalidate because project 4d51174af5fd4f069e42f35deb6ffead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:04.596668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.597760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-de95ca25-13fa-46ca-8f31-195284f32ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:04.600772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 942/1872] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:04 2022] DELETE /identity/v3/projects/4d51174af5fd4f069e42f35deb6ffead => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:04.607465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563baeb-94ff-4867-86f1-b8ebf8d8d8f6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.607639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7fa1-ce71-463d-a6ad-1c959c5fca1c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.608180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563baeb-94ff-4867-86f1-b8ebf8d8d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.608928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563baeb-94ff-4867-86f1-b8ebf8d8d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.613253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.613948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.614303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.614852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.615572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.617700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.639087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1563baeb-94ff-4867-86f1-b8ebf8d8d8f6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:04.641581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1563baeb-94ff-4867-86f1-b8ebf8d8d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.659149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 943/1873] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:04.677651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.678121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.678394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.699747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:04.701195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.714125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 944/1874] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:04.714311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.724600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f4559-1a75-466f-8315-7840ed39f8f2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.730795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.730980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.731221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.731499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.732159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:04.732456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.734215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.748202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 945/1875] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:04.762522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.773133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b25d9-0917-4fec-ac49-f9b6e6fd0804 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.778294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.778976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.779264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.779605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.780248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.782496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.789027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 931/1876] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/projects => generated 369 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:04.793219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.795559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:04.875570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0ff38-1cae-4235-b79b-bad9e8af3073 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.880896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.881626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.881953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.882139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.882711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.885279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.898319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 946/1877] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:04.900285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.977335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc77e3-f6fd-4f11-8dd5-fb4a5e5cd2e0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.984639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.985990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 932/1878] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:04.986940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.987991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.989258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.989549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.993621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.996405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.999492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faa856-60c0-4566-9a00-de8a5b801339 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.005008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.005638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.005923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.006596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.007136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.010739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.025737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 933/1879] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:05.029497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.041756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae833f-22fd-4fe3-97a8-d60d99afac44 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.047128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.047736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.048051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.048317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.052680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.052680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.063186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 934/1880] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:05.067312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.078590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e513c1-407f-40e0-83d7-2dfb23d12452 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.085163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.085831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.086212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.086507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] PATH_INFO: `/v3/projects/c4091d431ab94f5bb1df344679e7dc5d/users/6099543aaeea4207be8eae8cb404b054/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.103269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4091d431ab94f5bb1df344679e7dc5d, user_id=6099543aaeea4207be8eae8cb404b054, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.105232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.129278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 947/1881] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:05.132941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.135551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.149632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a178f-3772-47c8-8261-69742799dea2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.154942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.155364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.155648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.155923 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] PATH_INFO: `/v3/projects/3dba6764ecc0480fb4f4e1f2c533e2a8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.157098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dba6764ecc0480fb4f4e1f2c533e2a8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.159261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.207963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 935/1882] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:05 2022] PUT /identity/v3/projects/c4091d431ab94f5bb1df344679e7dc5d/users/6099543aaeea4207be8eae8cb404b054/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:05.212496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.276538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a14f1-c197-47c6-a1ff-3e3b56406049 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.289865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dba6764ecc0480fb4f4e1f2c533e2a8', 'request_id': 'req-a3451ac6-a01a-47af-86d6-419cc75a34a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:05.295835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dba6764ecc0480fb4f4e1f2c533e2a8', 'request_id': 'req-a3451ac6-a01a-47af-86d6-419cc75a34a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:05.299235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.304729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dba6764ecc0480fb4f4e1f2c533e2a8', 'request_id': 'req-a3451ac6-a01a-47af-86d6-419cc75a34a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:05.338304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e951d-8aa3-4029-ae3b-ec1974d6999f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.338930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e951d-8aa3-4029-ae3b-ec1974d6999f admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.339215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e951d-8aa3-4029-ae3b-ec1974d6999f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.339492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e951d-8aa3-4029-ae3b-ec1974d6999f admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.340144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e951d-8aa3-4029-ae3b-ec1974d6999f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.341832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e951d-8aa3-4029-ae3b-ec1974d6999f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.352651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 936/1883] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:05.354158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.354469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.354743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.381760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] MFA Rules not processed for user `6099543aaeea4207be8eae8cb404b054`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:05.382946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.391711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] The token cache is being invalidate because project 3dba6764ecc0480fb4f4e1f2c533e2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:05.392032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:05.392770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:05.397798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 948/1884] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:05 2022] DELETE /identity/v3/projects/3dba6764ecc0480fb4f4e1f2c533e2a8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:05.397798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.397798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.412711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3451ac6-a01a-47af-86d6-419cc75a34a0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.418450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de727ea-4bb5-45aa-ba31-e6c20669e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.419238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de727ea-4bb5-45aa-ba31-e6c20669e316 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.422482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de727ea-4bb5-45aa-ba31-e6c20669e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.422482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de727ea-4bb5-45aa-ba31-e6c20669e316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.422482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de727ea-4bb5-45aa-ba31-e6c20669e316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.422482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de727ea-4bb5-45aa-ba31-e6c20669e316 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.432766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 949/1885] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:05.434061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.434354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.434629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.455983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:05.457367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.467227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 937/1886] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:05.483451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 950/1887] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:05.483577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.483677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.483677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.486486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080b334-592b-479b-bf7b-4c6a6bce4881 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.488709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba2e44-b990-4fd9-90aa-544089ae4e0e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.492471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0412c43-c84f-45c0-805d-e32344ce797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.492873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0412c43-c84f-45c0-805d-e32344ce797a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.493199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0412c43-c84f-45c0-805d-e32344ce797a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.493570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0412c43-c84f-45c0-805d-e32344ce797a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.494212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0412c43-c84f-45c0-805d-e32344ce797a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.495021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.495714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.496509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.496633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0412c43-c84f-45c0-805d-e32344ce797a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.496724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] PATH_INFO: `/v3/projects/2e2109a4ee984f5ba1cb65b950b8d2dc/users/a47d7e44fd0542f4b0d44f63d223b38b/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.514935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2109a4ee984f5ba1cb65b950b8d2dc, user_id=a47d7e44fd0542f4b0d44f63d223b38b, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.517811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.558278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 951/1888] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:05.560416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5042c-c9c5-4b27-b0ca-af65a1275302 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.560753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5042c-c9c5-4b27-b0ca-af65a1275302 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.561058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5042c-c9c5-4b27-b0ca-af65a1275302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.582521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 938/1889] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:05 2022] PUT /identity/v3/projects/2e2109a4ee984f5ba1cb65b950b8d2dc/users/a47d7e44fd0542f4b0d44f63d223b38b/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:05.582521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.584077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a5e5042c-c9c5-4b27-b0ca-af65a1275302 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:05.585506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a5e5042c-c9c5-4b27-b0ca-af65a1275302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.653264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fb41-41bc-4993-9558-efb0f139bcee admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.658544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.659169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.659452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.659727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.660215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 952/1890] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:05.661719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.662096 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.662468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.663567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.666987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.693791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] MFA Rules not processed for user `a47d7e44fd0542f4b0d44f63d223b38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:05.695171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.744437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 939/1891] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:05.745654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.757174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc1209-4d38-47f9-83a7-77c6f704ad4a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.763401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.764215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.764635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.765041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.765726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:05.766034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.767731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.772828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 953/1892] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:05.772828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.785453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff508a0-4ac5-4dd7-a186-6b840cc9eac6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.787825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 940/1893] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:05.789421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.792337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.793590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.793872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.794145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.794982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.796603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.807725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 954/1894] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:05.808540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.814980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a13c01-440b-41d7-9813-d12526bf05e3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.820401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2765b-4a5a-4949-a34b-8d036604f22f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.823245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.824033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.825672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.825672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.825672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.826484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.827100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.827231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.827433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.827714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.828336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.830356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.953697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 955/1895] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:05.954689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.983682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 941/1896] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:05.983682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.077560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bec95-eca3-4031-9774-62010be38ba5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.083406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.083979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4294351-b5ac-4765-8e4a-9707c273c142 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.084107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.088783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.088783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] PATH_INFO: `/v3/projects/790865ec5baf436095b1fecdfdc10ab1/users/959b439d75ed4907a8e3f7602977a5d5/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.089609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.090217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.090497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.090767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.091390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.093533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.103589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790865ec5baf436095b1fecdfdc10ab1, user_id=959b439d75ed4907a8e3f7602977a5d5, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.106471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.200486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:06.206527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 956/1897] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:05 2022] PUT /identity/v3/projects/790865ec5baf436095b1fecdfdc10ab1/users/959b439d75ed4907a8e3f7602977a5d5/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:06.207356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.220277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 942/1898] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:06.221316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.224116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.237285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7a1b9-d61e-4ce5-b441-572540147a5b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.296122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92029595-d8a3-4fec-8d8b-7d29bca3ebb8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.301706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.302338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.302617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.302936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.303572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.306279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.311559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.312533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.312916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.313374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] PATH_INFO: `/v3/users/7a1212ff4b5c4c7aa6b4a04589c1c559` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.315382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a1212ff4b5c4c7aa6b4a04589c1c559)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.317555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.405044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 957/1899] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:06 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:06.405901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.417798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476361a1-0c00-4c7e-ad6d-e455a03187c6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.424181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.425036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.425476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.425922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.426843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.428894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.450082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 958/1900] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:06.451124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.462204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc05bc-686f-4cb0-b3cc-2a5a25d653dd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.467336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.467978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.468260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.468532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.469208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.470801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.482568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 959/1901] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:06.483558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.494485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1fd02-dffa-4d2b-987d-4f74155f3e58 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.500388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.501026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.501328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.501601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.502245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.504116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.521446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a1212ff4b5c4c7aa6b4a04589c1c559'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:06.522057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 960/1902] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:06.522057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.529757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160104dd-7433-4e3a-893c-f0bde11cac29 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.534717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.538620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.538620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.538620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] PATH_INFO: `/v3/projects/790865ec5baf436095b1fecdfdc10ab1/users/959b439d75ed4907a8e3f7602977a5d5/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.538620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=790865ec5baf436095b1fecdfdc10ab1, user_id=959b439d75ed4907a8e3f7602977a5d5, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.540844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.551217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 961/1903] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:06 2022] PUT /identity/v3/projects/790865ec5baf436095b1fecdfdc10ab1/users/959b439d75ed4907a8e3f7602977a5d5/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:06.552011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.596207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a1212ff4b5c4c7aa6b4a04589c1c559'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:06.606215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a1212ff4b5c4c7aa6b4a04589c1c559'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:06.607032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a1212ff4b5c4c7aa6b4a04589c1c559'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:06.622897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a1212ff4b5c4c7aa6b4a04589c1c559'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:06.624687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d7370-40a7-4b51-94d6-3653f144af8d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.629876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7466694-7d03-403f-9a84-3463381ff647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.631848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7466694-7d03-403f-9a84-3463381ff647 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.631848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7466694-7d03-403f-9a84-3463381ff647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.632192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7466694-7d03-403f-9a84-3463381ff647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.632624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 943/1904] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:06 2022] DELETE /identity/v3/users/7a1212ff4b5c4c7aa6b4a04589c1c559 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:06.633009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7466694-7d03-403f-9a84-3463381ff647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.634380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.636412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.637988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:06.638574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7466694-7d03-403f-9a84-3463381ff647 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.649576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 962/1905] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:06.653932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099e44-7bcf-41e9-bd5a-2016f3535364 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.654999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099e44-7bcf-41e9-bd5a-2016f3535364 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.655282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099e44-7bcf-41e9-bd5a-2016f3535364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.687678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-29099e44-7bcf-41e9-bd5a-2016f3535364 None None] MFA Rules not processed for user `959b439d75ed4907a8e3f7602977a5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:06.688945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-29099e44-7bcf-41e9-bd5a-2016f3535364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.717472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d8f32-9f9a-49e5-a5a4-8c39a5a7d40d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.722432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.723224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.723677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.724059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] PATH_INFO: `/v3/projects/37d6daab08404f2db0da883c346d4453/users/33165e00f6474776aae30e5b8597c889/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.741302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d6daab08404f2db0da883c346d4453, user_id=33165e00f6474776aae30e5b8597c889, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.744737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.765984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 963/1906] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:06.767243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.767522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.767794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.788632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:06.790521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.798617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 964/1907] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:06.799376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.800748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.812212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caa115-ecf2-4a96-b598-fe99e948525c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.816757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.828375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.828375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.828375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] PATH_INFO: `/v3/projects/c5d1aa9f1b02459f87648af68cc13cdf/users/2bf3fce67d7b4068ac62d4f41ba6d8a4/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.832188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d1aa9f1b02459f87648af68cc13cdf, user_id=2bf3fce67d7b4068ac62d4f41ba6d8a4, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.834896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.853621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 944/1908] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:06 2022] PUT /identity/v3/projects/37d6daab08404f2db0da883c346d4453/users/33165e00f6474776aae30e5b8597c889/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:06.854456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.855973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.910822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598cf0e-2f60-4f0d-a82e-6be0dfbb9f07 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.914547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.915339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.915624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.915931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.928101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 965/1909] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:06 2022] PUT /identity/v3/projects/c5d1aa9f1b02459f87648af68cc13cdf/users/2bf3fce67d7b4068ac62d4f41ba6d8a4/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:06.937931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.939682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.977601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.979208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.990655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 945/1910] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:06.991400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.008954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072736e9-a390-456d-b4f9-e21b2769ee41 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.013599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f6e10-c2c7-4d25-9db4-355322e932d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.014227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6e10-c2c7-4d25-9db4-355322e932d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.014512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6e10-c2c7-4d25-9db4-355322e932d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.014786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6e10-c2c7-4d25-9db4-355322e932d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.061743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6251-c546-41c9-b36f-978ed761e381 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.103785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6e10-c2c7-4d25-9db4-355322e932d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.104255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6e10-c2c7-4d25-9db4-355322e932d2 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.115284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 966/1911] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:07.116688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.117120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.117564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.155870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.157063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.157499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.158391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.158980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] MFA Rules not processed for user `2bf3fce67d7b4068ac62d4f41ba6d8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:07.160237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.160397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:07.162285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.184034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 946/1912] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:07.184980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.187217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:07.260312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 967/1913] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:07.261817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.272803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc70e6-61a6-4931-b7cf-0165e3c669ef admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.274327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc880bcc-6f67-4f30-9abf-b34c94944936 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.278078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.278697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.278980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.279721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.280079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.280873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.281267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.286352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] PATH_INFO: `/v3/projects/37d6daab08404f2db0da883c346d4453/users/33165e00f6474776aae30e5b8597c889/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.286691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:07.286782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d6daab08404f2db0da883c346d4453, user_id=33165e00f6474776aae30e5b8597c889, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.288015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.299020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 968/1914] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:07 2022] PUT /identity/v3/projects/37d6daab08404f2db0da883c346d4453/users/33165e00f6474776aae30e5b8597c889/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:07.299856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.350138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.352256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca85230-36db-47b7-a4fd-f8eadf5d71cf service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.363911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 947/1915] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:07.365260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.365549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.365824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.390860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8ace8-271e-42aa-be21-7d7db1e394f6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.401893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.401893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.401893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.401893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.402232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.402232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.404906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] MFA Rules not processed for user `33165e00f6474776aae30e5b8597c889`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:07.406165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:07.500565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 948/1916] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:07.515996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.518573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 969/1917] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:07 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:07.523152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.631840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:07.637752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d4cc0-d291-4686-95ef-0ade1672bde1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.656077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d56df5-89c3-4419-8243-a4741bad9ff5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.661674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.662296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.662583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.662973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.663614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.666142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.708831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.708831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.708831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.708831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.711982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:07.762542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 970/1918] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:07 2022] POST /identity/v3/users => generated 294 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:07.766492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.769667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:07.794783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fa6c-dc1b-401d-bff4-f3f30faffa0f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.822877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.823876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.824259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.824602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] PATH_INFO: `/v3/projects/c3e2fda9c8d94db8a3e6794ebf630793` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.826583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e2fda9c8d94db8a3e6794ebf630793)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.828605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.830689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.832430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.846947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 949/1919] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:07.848033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.853874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:07.861106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cec4bf-1c76-478e-8a1f-0b66523bd57d service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.865941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.869279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.869279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.869279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.869279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.870092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.881781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 950/1920] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:07.891596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.910555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252ff9f-9879-4ecd-8234-9732589be1ee admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.919501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.920205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.920609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.920766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.922852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.925603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.948605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 951/1921] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:07.949540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.959863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e2fda9c8d94db8a3e6794ebf630793', 'request_id': 'req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:07.960496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf23af2-1a4b-4fe3-9655-a07e369c10cc admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.967276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e2fda9c8d94db8a3e6794ebf630793', 'request_id': 'req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:07.978138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e2fda9c8d94db8a3e6794ebf630793', 'request_id': 'req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.022128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199771-c346-4cdb-ba5a-b876e2f59c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.022710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199771-c346-4cdb-ba5a-b876e2f59c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.023057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199771-c346-4cdb-ba5a-b876e2f59c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.023347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199771-c346-4cdb-ba5a-b876e2f59c88 admin admin] PATH_INFO: `/v3/projects/9c5c741be3d542c08ddd8519d0d3e89a/users/3c44b046f276448a945ad167ff414232/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.040795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199771-c346-4cdb-ba5a-b876e2f59c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5c741be3d542c08ddd8519d0d3e89a, user_id=3c44b046f276448a945ad167ff414232, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.044818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199771-c346-4cdb-ba5a-b876e2f59c88 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.099391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] The token cache is being invalidate because project c3e2fda9c8d94db8a3e6794ebf630793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:08.099391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.099391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.103643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 971/1922] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:07 2022] DELETE /identity/v3/projects/c3e2fda9c8d94db8a3e6794ebf630793 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:08.104554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.106827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.120339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeeaf3-bff3-4a6e-9de1-7f7592e058db admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.122265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 952/1923] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:07 2022] PUT /identity/v3/projects/9c5c741be3d542c08ddd8519d0d3e89a/users/3c44b046f276448a945ad167ff414232/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:08.138579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.138875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.139181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.172400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] MFA Rules not processed for user `3c44b046f276448a945ad167ff414232`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:08.173792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.186437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.187090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.187457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.190696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] PATH_INFO: `/v3/users/e9ebefbe68b64ddd935a19fa2a3fef28` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.190696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ebefbe68b64ddd935a19fa2a3fef28)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.190979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.321976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 953/1924] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:08.326411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.334225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.358331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f38d6-219f-4e9e-a59a-40a02bd4fb52 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.364170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.364925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.365266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.371692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.371692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.371692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.419207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ebefbe68b64ddd935a19fa2a3fef28'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.428095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ebefbe68b64ddd935a19fa2a3fef28'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.476018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 954/1925] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:08 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:08.478137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.510802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ebefbe68b64ddd935a19fa2a3fef28'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.512174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ebefbe68b64ddd935a19fa2a3fef28'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.528106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ebefbe68b64ddd935a19fa2a3fef28'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:08.534956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 972/1926] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:08 2022] DELETE /identity/v3/users/e9ebefbe68b64ddd935a19fa2a3fef28 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:08.565924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19919bf2-3f77-40c0-9619-433bfd8addad admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.683484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.684229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.684594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.685268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.685906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.687835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.764925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 955/1927] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:08 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:08.768793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.770615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.793489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b66730-64d3-4f2b-93ce-68c21c1e2249 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.797201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.797969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.798296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.798638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.799509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.801657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.816141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 973/1928] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:08.824823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.836910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c374f3-ceec-4bc3-b3c9-f1edc5317c28 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.843304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.844303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.849678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.849678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] PATH_INFO: `/v3/projects/5202c563a0c249bfb68885e7f8ad4c63/users/a4d36d7fbb8747d28be0babc23f5242c/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.863804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5202c563a0c249bfb68885e7f8ad4c63, user_id=a4d36d7fbb8747d28be0babc23f5242c, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.869282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.959870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 956/1929] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:08 2022] PUT /identity/v3/projects/5202c563a0c249bfb68885e7f8ad4c63/users/a4d36d7fbb8747d28be0babc23f5242c/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:08.964075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.998615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a0216-e68f-4cdc-baa3-b0837cd60cda admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.072095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.087507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfa2a2-d657-47c7-afbc-6c584aae5beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.088269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfa2a2-d657-47c7-afbc-6c584aae5beb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.088624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfa2a2-d657-47c7-afbc-6c584aae5beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.089049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfa2a2-d657-47c7-afbc-6c584aae5beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.089767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfa2a2-d657-47c7-afbc-6c584aae5beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.093146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfa2a2-d657-47c7-afbc-6c584aae5beb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.105098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 974/1930] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:09.106291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.118879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947394be-ff95-4979-b308-e9b10cd6560b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.123721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.124509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.124856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.125230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] PATH_INFO: `/v3/projects/5202c563a0c249bfb68885e7f8ad4c63/users/a4d36d7fbb8747d28be0babc23f5242c/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.127221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5202c563a0c249bfb68885e7f8ad4c63, user_id=a4d36d7fbb8747d28be0babc23f5242c, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.130021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.143088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 957/1931] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:09 2022] PUT /identity/v3/projects/5202c563a0c249bfb68885e7f8ad4c63/users/a4d36d7fbb8747d28be0babc23f5242c/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:09.154781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.155073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.155353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.188726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] MFA Rules not processed for user `a4d36d7fbb8747d28be0babc23f5242c`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:09.189985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:09.201806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.208857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.210770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:09.291279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 975/1932] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:09.302507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.306257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:09.307116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ede5-f742-4fc8-87d4-ad263287c22a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.313033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.314111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.316490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.316910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] PATH_INFO: `/v3/projects/264cb9191d9e49d8b1781e943ad9c137` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.318604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=264cb9191d9e49d8b1781e943ad9c137)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.320859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.376210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a669dcc-2960-40b7-9ced-c560d9eb959b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.380866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd09286-ccb2-4a30-9f2e-318e36b50703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.381700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd09286-ccb2-4a30-9f2e-318e36b50703 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.381982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd09286-ccb2-4a30-9f2e-318e36b50703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.382257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd09286-ccb2-4a30-9f2e-318e36b50703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.393158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd09286-ccb2-4a30-9f2e-318e36b50703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.395861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd09286-ccb2-4a30-9f2e-318e36b50703 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.416680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264cb9191d9e49d8b1781e943ad9c137', 'request_id': 'req-6d4d0098-f64f-4b70-812e-e09ce3e68986'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:09.422149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 976/1933] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:09.423536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264cb9191d9e49d8b1781e943ad9c137', 'request_id': 'req-6d4d0098-f64f-4b70-812e-e09ce3e68986'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:09.433758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264cb9191d9e49d8b1781e943ad9c137', 'request_id': 'req-6d4d0098-f64f-4b70-812e-e09ce3e68986'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:09.517172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] The token cache is being invalidate because project 264cb9191d9e49d8b1781e943ad9c137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:09.517685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:09.519223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d4d0098-f64f-4b70-812e-e09ce3e68986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:09.521988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 958/1934] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:09 2022] DELETE /identity/v3/projects/264cb9191d9e49d8b1781e943ad9c137 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:09.528294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.529049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.529475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.560954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:09.563301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:09.579237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.631225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.647753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 977/1935] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:09.662446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.662446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.662758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.719934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:09.727241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:09.736558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 959/1936] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:09.742810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.753005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b520-de6b-4ce4-8034-e8eccd1e95b5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.758357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.759393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.759677 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.759946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.760606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:09.760909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.762574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.790005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 978/1937] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:09.791691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.806306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31dd67-e276-46b7-a0d5-0b59f58099e9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.814278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.814977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.815267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.815899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.816972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.819437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.935693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 960/1938] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/projects => generated 361 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:09.939783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.950410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474054d-63ef-4faf-b804-819ea65f96aa admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.990618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.022132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.022716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.023000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.023275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.024625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.026713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.070351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.157419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 979/1939] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/users => generated 281 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:10.161703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.182590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917eab5-f944-4bea-8d47-1d71f388e1a7 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.189866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.190493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.190778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.191053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.191713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.193374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.208274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 961/1940] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:10.212348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.232148 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6d14b-2765-47c9-8152-618448111243 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.237252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece130f7-d7aa-477e-9e7a-6f10ab26ea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.238178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece130f7-d7aa-477e-9e7a-6f10ab26ea4c admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.238495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece130f7-d7aa-477e-9e7a-6f10ab26ea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.238827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece130f7-d7aa-477e-9e7a-6f10ab26ea4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.239470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece130f7-d7aa-477e-9e7a-6f10ab26ea4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.241080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece130f7-d7aa-477e-9e7a-6f10ab26ea4c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.253418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 980/1941] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:10.257679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.267969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c58e62-2e50-4956-9faa-e612f7afd013 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.272731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.274956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.275232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.275505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] PATH_INFO: `/v3/projects/666a141bbde740e0900e32940bc2f96f/users/52285fa325454944a6f2910d45f2873b/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.291336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=666a141bbde740e0900e32940bc2f96f, user_id=52285fa325454944a6f2910d45f2873b, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.294026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.398772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 962/1942] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:10 2022] PUT /identity/v3/projects/666a141bbde740e0900e32940bc2f96f/users/52285fa325454944a6f2910d45f2873b/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:10.410838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.411140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.411387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.443546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] MFA Rules not processed for user `52285fa325454944a6f2910d45f2873b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:10.444767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:10.552942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 981/1943] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:10.778843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.780790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:10.818963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.864921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc03db-cd96-4271-b3ac-40c6b8f8d72d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.869323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.870069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.870307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.870575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.879763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.881260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.895537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 963/1944] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:10.915752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.011825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573283b-0929-4fe0-bec7-92fc0ddd6af0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.019760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.020554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.020938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.021317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] PATH_INFO: `/v3/users/52285fa325454944a6f2910d45f2873b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.022559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52285fa325454944a6f2910d45f2873b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.025270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.048697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.059464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00194fd5-21bb-46e9-8dda-f30773c61482 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.067351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.068089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.068912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.069452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.073485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:11.165183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.166718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb075f-5678-467d-a5d6-b27afa6ede8a service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.186857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 964/1945] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:11.191175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:11.314486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52285fa325454944a6f2910d45f2873b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:11.329841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52285fa325454944a6f2910d45f2873b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:11.402659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52285fa325454944a6f2910d45f2873b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:11.403326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52285fa325454944a6f2910d45f2873b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:11.425127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52285fa325454944a6f2910d45f2873b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:11.441992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 982/1946] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:10 2022] DELETE /identity/v3/users/52285fa325454944a6f2910d45f2873b => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:11.446550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.446838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.447122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.471887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:11.473158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:11.609680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 965/1947] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:11.616682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.621968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:11.766603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3f9c-e27b-450a-bcf0-51a79f9b866d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.771123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735db4f-13db-493d-98fe-1169ee14fde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.772002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735db4f-13db-493d-98fe-1169ee14fde1 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.772460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735db4f-13db-493d-98fe-1169ee14fde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.772964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735db4f-13db-493d-98fe-1169ee14fde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.785842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735db4f-13db-493d-98fe-1169ee14fde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.787650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735db4f-13db-493d-98fe-1169ee14fde1 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.801149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 983/1948] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:11.860120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:11.922369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.202188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.470543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.563843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:12.569146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:12.584564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434f460-5a7e-40c7-a14b-dc553c76031d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:12.590931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:12.591567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.591843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.592319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] PATH_INFO: `/v3/projects/666a141bbde740e0900e32940bc2f96f` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.593514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] RBAC: Authorizing `identity:delete_project(project_id=666a141bbde740e0900e32940bc2f96f)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:12.595730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:12.748669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666a141bbde740e0900e32940bc2f96f', 'request_id': 'req-e285ffce-09c2-4b42-adfa-b60dde989daa'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:12.755309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666a141bbde740e0900e32940bc2f96f', 'request_id': 'req-e285ffce-09c2-4b42-adfa-b60dde989daa'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:12.763548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666a141bbde740e0900e32940bc2f96f', 'request_id': 'req-e285ffce-09c2-4b42-adfa-b60dde989daa'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:12.779049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.873112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.879415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] The token cache is being invalidate because project 666a141bbde740e0900e32940bc2f96f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:12.879709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:12.880367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e285ffce-09c2-4b42-adfa-b60dde989daa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:12.887295 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 966/1949] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:12 2022] DELETE /identity/v3/projects/666a141bbde740e0900e32940bc2f96f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:12.889056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.889585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.890078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.929397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:12.931480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.054108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 984/1950] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:13.062904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.068620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.068620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.087399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:13.088505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:13.088650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.096855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 967/1951] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:13.101243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.113635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368d12e-2623-46c2-bbd4-abcb12ec0fa6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.120156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.120787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.122471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.122471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.122471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:13.122471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.124680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.141161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 985/1952] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:13.162305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.192852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959428-bb83-476f-bdd8-8bc3162433e9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.200968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.205200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.207511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.207928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.210237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.213767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.358246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 968/1953] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:13.365573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.389068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bcf26-17e4-467c-8f58-17ebf72bd512 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.442604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.446377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.471086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.472540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.472806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.473412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.485201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.485201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.536700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072ea7e-559a-4542-96cb-b289eb22e2ee admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.542563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.543293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.543569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.544201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] PATH_INFO: `/v3/users/6099543aaeea4207be8eae8cb404b054` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.545914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6099543aaeea4207be8eae8cb404b054)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.548035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.592323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 986/1954] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:13.596829 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.611636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a0cf-db2a-4f1c-8397-7c99af069e7c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.620028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.620684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.620965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.621317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.621932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.623597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.636071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 987/1955] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:13.640257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.651618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9b9bb-a6b6-4454-bf98-5be9fa7c8fda admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.658375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.659772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.659772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.659772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.660146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.662040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.675455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 988/1956] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:13.686477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.700779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79f618-ec50-4475-b93d-57e866db4c27 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.708737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675ea23-aca1-4ce1-be85-9bad6d58abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.712646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675ea23-aca1-4ce1-be85-9bad6d58abec admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.712646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675ea23-aca1-4ce1-be85-9bad6d58abec admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.712646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675ea23-aca1-4ce1-be85-9bad6d58abec admin admin] PATH_INFO: `/v3/projects/6cbae98fb64443d8b14f7a45d1a659cc/users/e8b2e816014042a380782d72f9d2ccb2/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.729410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675ea23-aca1-4ce1-be85-9bad6d58abec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbae98fb64443d8b14f7a45d1a659cc, user_id=e8b2e816014042a380782d72f9d2ccb2, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.732305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675ea23-aca1-4ce1-be85-9bad6d58abec admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.780907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6099543aaeea4207be8eae8cb404b054'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:13.809182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 989/1957] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:13 2022] PUT /identity/v3/projects/6cbae98fb64443d8b14f7a45d1a659cc/users/e8b2e816014042a380782d72f9d2ccb2/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:13.821286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.821604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.821882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.849404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] MFA Rules not processed for user `e8b2e816014042a380782d72f9d2ccb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:13.850625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.875737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6099543aaeea4207be8eae8cb404b054'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:13.884182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6099543aaeea4207be8eae8cb404b054'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:13.884860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6099543aaeea4207be8eae8cb404b054'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:13.897239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55442af0-267e-4fe2-8183-a37f3e1cd631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6099543aaeea4207be8eae8cb404b054'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:13.904104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 969/1958] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:13 2022] DELETE /identity/v3/users/6099543aaeea4207be8eae8cb404b054 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:13.908898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.909288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.909596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.918862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 990/1959] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:13.929838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.930853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:13.931319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.932112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.956264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:13.990012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40fbb-f1a8-4ca9-b924-e5ba374cb360 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.994523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 970/1960] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:14.004977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.006775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:14.020833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.045570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.046553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.046828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.047098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.066409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744951c2-68b7-43b4-8f81-2dea26454016 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.070233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.071892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.071892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.072141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.082890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.084548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.095648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 971/1961] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:14.107522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.109031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.119941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 991/1962] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:14.201917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.454359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.511382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.525771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0147a-d2ca-49a7-85ec-5af0d960b4c7 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.529684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.530324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.530613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.530944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.532801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:14.578534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.580159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:14.603649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc391b-5ff6-4af2-b9b0-77b3f5064d6c service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.612886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.613059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.613389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.613731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] PATH_INFO: `/v3/projects/c4091d431ab94f5bb1df344679e7dc5d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.614968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4091d431ab94f5bb1df344679e7dc5d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.618280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.623229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.639409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.651224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 972/1963] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:14.741489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4091d431ab94f5bb1df344679e7dc5d', 'request_id': 'req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:14.750165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4091d431ab94f5bb1df344679e7dc5d', 'request_id': 'req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:14.758893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4091d431ab94f5bb1df344679e7dc5d', 'request_id': 'req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:14.847727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] The token cache is being invalidate because project c4091d431ab94f5bb1df344679e7dc5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:14.848418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:14.848934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:14.850637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 992/1964] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:14 2022] DELETE /identity/v3/projects/c4091d431ab94f5bb1df344679e7dc5d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:14.854951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.868946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfb5a8-3d4c-4a42-bd73-aed6db0eb324 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.960152 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.960920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.961327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.961714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] PATH_INFO: `/v3/users/959b439d75ed4907a8e3f7602977a5d5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.963106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=959b439d75ed4907a8e3f7602977a5d5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.965682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:15.165302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '959b439d75ed4907a8e3f7602977a5d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:15.247371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '959b439d75ed4907a8e3f7602977a5d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:15.256190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '959b439d75ed4907a8e3f7602977a5d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:15.256880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '959b439d75ed4907a8e3f7602977a5d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:15.266832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:15.280356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7c945d70-6d0f-4b54-bf08-9281f5bed4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '959b439d75ed4907a8e3f7602977a5d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:15.287124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 973/1965] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:14 2022] DELETE /identity/v3/users/959b439d75ed4907a8e3f7602977a5d5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:15.821553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:15.824435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:15.838810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da11aa-bbbb-4790-b2ab-8f16f91102b0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:15.902872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:15.910793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:15.911691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:15.912138 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:15.912753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] PATH_INFO: `/v3/projects/790865ec5baf436095b1fecdfdc10ab1` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:15.914904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=790865ec5baf436095b1fecdfdc10ab1)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:15.917275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.064983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790865ec5baf436095b1fecdfdc10ab1', 'request_id': 'req-17d54f1c-d842-454f-92b7-03d461b34f67'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:16.071198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790865ec5baf436095b1fecdfdc10ab1', 'request_id': 'req-17d54f1c-d842-454f-92b7-03d461b34f67'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:16.078284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790865ec5baf436095b1fecdfdc10ab1', 'request_id': 'req-17d54f1c-d842-454f-92b7-03d461b34f67'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:16.173645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] The token cache is being invalidate because project 790865ec5baf436095b1fecdfdc10ab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:16.173939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:16.174553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-17d54f1c-d842-454f-92b7-03d461b34f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:16.177344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 993/1966] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:15 2022] DELETE /identity/v3/projects/790865ec5baf436095b1fecdfdc10ab1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:16.183432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.183925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.184044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.209466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:16.210696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:16.297669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 974/1967] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:16.307227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.307636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.308012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.335513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:16.337981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:16.346800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 994/1968] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:16.351338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.362550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5c04-3b0d-4988-879a-6c9618a56d47 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.371426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.372139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.372674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.373137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.373823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:16.374162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.376224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.390710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 975/1969] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:16.402989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.411473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b25610-486d-42dd-9e9b-619f5a33e570 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.420059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.420979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.421537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.421868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.422722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.425113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.559115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 995/1970] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/projects => generated 409 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:16.563154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.575139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db700c2d-6357-474b-bdff-6af76d11ebf9 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.597111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:16.652463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.652463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.652463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.652463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.652871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.653303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.755445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 976/1971] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/users => generated 305 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:16.756399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.786514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98c0d-1b9b-45c2-a436-6c7b88d69d67 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.801456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.802095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.802370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.802653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.803390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.805068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.827405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 996/1972] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:16.830870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.843124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2e60-404b-4f79-8dd8-475cc8d439bd admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.859570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45466-3de6-4af9-98f4-8390b45516ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.860209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45466-3de6-4af9-98f4-8390b45516ba admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.861452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45466-3de6-4af9-98f4-8390b45516ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.861601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45466-3de6-4af9-98f4-8390b45516ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.862300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd45466-3de6-4af9-98f4-8390b45516ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.869355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd45466-3de6-4af9-98f4-8390b45516ba admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.900958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 977/1973] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:16.916652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.937163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcbc66-d6c8-4dda-82f2-3d6b3a755602 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.940426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.941259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.954911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.955299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] PATH_INFO: `/v3/projects/ae61ce475940488883ddf66b78edb2cf/users/a0247e2771c94cf984ccd89dd935e658/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.978626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae61ce475940488883ddf66b78edb2cf, user_id=a0247e2771c94cf984ccd89dd935e658, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.981501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.071226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 997/1974] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:16 2022] PUT /identity/v3/projects/ae61ce475940488883ddf66b78edb2cf/users/a0247e2771c94cf984ccd89dd935e658/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:17.089342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.089342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.089342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.150102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] MFA Rules not processed for user `a0247e2771c94cf984ccd89dd935e658`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:17.151338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:17.240707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 978/1975] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:17.256416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.334340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38570-13cd-4c6c-86c5-2dbc6f61c148 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.343232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.343232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.343232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.343232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.343574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.343574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.380392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.382472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:17.464950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d20dc-09c4-42d2-8973-ebf8909698b0 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.467360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 998/1976] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/projects => generated 409 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:17.472305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.485803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:17.536094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:17.543528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.544216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.544440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.544713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.548172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:17.562858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8912b-2b1d-4af1-8bd8-2d6acc849b0d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.569562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.571139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.571686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.572324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.573137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.575416 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.618243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.619779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.629649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 979/1977] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:17.680625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 999/1978] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/users => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:17.681912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.699279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cf18a-4e83-4643-8342-9eecd35bf9d1 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.706504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.707398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.707776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.708052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.708761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.711875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.730160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 980/1979] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:17.731870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.745768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdec544-955c-49bc-ae20-e2ffabeff73b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.750537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.751319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.751606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.751876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] PATH_INFO: `/v3/projects/122c3470c77643f09b0243ac6563a296/users/25c6c6881a0d45d1b48bfd8eaea6c18c/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.768917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] RBAC: Authorizing `identity:create_grant(project_id=122c3470c77643f09b0243ac6563a296, user_id=25c6c6881a0d45d1b48bfd8eaea6c18c, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.771715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.835679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1000/1980] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:17 2022] PUT /identity/v3/projects/122c3470c77643f09b0243ac6563a296/users/25c6c6881a0d45d1b48bfd8eaea6c18c/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:17.840340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.851404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da57b4-2250-4037-996a-fd81ba9d265e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.909390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f8ac3-65c1-4a45-83f8-6b323fa0d001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.910010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f8ac3-65c1-4a45-83f8-6b323fa0d001 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.910287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f8ac3-65c1-4a45-83f8-6b323fa0d001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.910560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f8ac3-65c1-4a45-83f8-6b323fa0d001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.911214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f8ac3-65c1-4a45-83f8-6b323fa0d001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.912811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f8ac3-65c1-4a45-83f8-6b323fa0d001 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.927363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 981/1981] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:17.928522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.940611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113f19-e532-40d1-afba-e5e690e7455d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.946689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.947461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.947738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.948019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] PATH_INFO: `/v3/projects/122c3470c77643f09b0243ac6563a296/users/25c6c6881a0d45d1b48bfd8eaea6c18c/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.950230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=122c3470c77643f09b0243ac6563a296, user_id=25c6c6881a0d45d1b48bfd8eaea6c18c, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.953514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.965959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1001/1982] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:17 2022] PUT /identity/v3/projects/122c3470c77643f09b0243ac6563a296/users/25c6c6881a0d45d1b48bfd8eaea6c18c/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:17.978359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282539e-3398-4966-9764-8644cab7abde None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.978654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282539e-3398-4966-9764-8644cab7abde None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.978933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282539e-3398-4966-9764-8644cab7abde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.007774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9282539e-3398-4966-9764-8644cab7abde None None] MFA Rules not processed for user `25c6c6881a0d45d1b48bfd8eaea6c18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:18.009321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9282539e-3398-4966-9764-8644cab7abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:18.088007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 982/1983] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:18.113991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:18.115675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:18.177901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18533-ba16-4d6e-afb0-b2abbbea9848 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:18.182269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:18.182884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:18.183155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:18.183425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.192264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:18.193704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:18.202948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1002/1984] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:18 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:19.248605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282539e-3398-4966-9764-8644cab7abde None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:19.260279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282539e-3398-4966-9764-8644cab7abde None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:19.264605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f6e5-7212-4d67-89aa-5646ea69a3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:19.265246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f6e5-7212-4d67-89aa-5646ea69a3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.265525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f6e5-7212-4d67-89aa-5646ea69a3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.265805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f6e5-7212-4d67-89aa-5646ea69a3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.276581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:19.338937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f6e5-7212-4d67-89aa-5646ea69a3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:19.340374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f6e5-7212-4d67-89aa-5646ea69a3d8 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:19.349989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 983/1985] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:19 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:22.063472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:22.068909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:22.135853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892e929-4c42-4a7b-9a42-7754fb831f1b service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:22.141317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:22.146958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.146958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.146958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] PATH_INFO: `/v3/users/a47d7e44fd0542f4b0d44f63d223b38b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.146958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47d7e44fd0542f4b0d44f63d223b38b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:22.146958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:22.151401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:22.336688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47d7e44fd0542f4b0d44f63d223b38b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.350403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47d7e44fd0542f4b0d44f63d223b38b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.419012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47d7e44fd0542f4b0d44f63d223b38b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.419705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47d7e44fd0542f4b0d44f63d223b38b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.445821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47d7e44fd0542f4b0d44f63d223b38b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.445821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1003/1986] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:22 2022] DELETE /identity/v3/users/a47d7e44fd0542f4b0d44f63d223b38b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:22.445990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.445990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.445990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.465821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:22.467128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:22.535846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 984/1987] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:22.545832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:22.548133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:22.548254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:22.549515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:22.563863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39cf52-b20b-4dec-9819-b6eb1b160f2c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:22.568589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:22.569257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.569541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.569813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] PATH_INFO: `/v3/users/2bf3fce67d7b4068ac62d4f41ba6d8a4` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.571233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf3fce67d7b4068ac62d4f41ba6d8a4)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:22.573236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:22.616362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0bd05-0ab9-4273-b858-cf78b88e4b93 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:22.625771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c2aac-378b-46b1-92b6-38bb7b13be96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:22.626537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c2aac-378b-46b1-92b6-38bb7b13be96 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.626923 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c2aac-378b-46b1-92b6-38bb7b13be96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.627297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c2aac-378b-46b1-92b6-38bb7b13be96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.637464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c2aac-378b-46b1-92b6-38bb7b13be96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:22.638926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c2aac-378b-46b1-92b6-38bb7b13be96 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:22.653448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1004/1988] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:22.745479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf3fce67d7b4068ac62d4f41ba6d8a4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.825959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf3fce67d7b4068ac62d4f41ba6d8a4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.836651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf3fce67d7b4068ac62d4f41ba6d8a4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.838114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf3fce67d7b4068ac62d4f41ba6d8a4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.853979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf3fce67d7b4068ac62d4f41ba6d8a4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:22.861031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 985/1989] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:22 2022] DELETE /identity/v3/users/2bf3fce67d7b4068ac62d4f41ba6d8a4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:22.866139 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.866384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.866666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.903476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:22.904734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:22.977886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1005/1990] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:22.990526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:22.992040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:23.029361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.066349 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a0e23-4052-4b22-bd40-0de93a904822 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.071472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.076944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.076944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.076944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.079616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.081094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:23.104704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee73c2-6602-4c24-afcc-93a535256f7d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.108938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.110279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.110590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.111847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.112232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.112408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] PATH_INFO: `/v3/projects/2e2109a4ee984f5ba1cb65b950b8d2dc` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.113586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2109a4ee984f5ba1cb65b950b8d2dc)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.115564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.135806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 986/1991] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:23.159654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.279130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2109a4ee984f5ba1cb65b950b8d2dc', 'request_id': 'req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.284987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2109a4ee984f5ba1cb65b950b8d2dc', 'request_id': 'req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.293953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2109a4ee984f5ba1cb65b950b8d2dc', 'request_id': 'req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.378756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] The token cache is being invalidate because project 2e2109a4ee984f5ba1cb65b950b8d2dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:23.379126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.380744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.383299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1006/1992] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:23 2022] DELETE /identity/v3/projects/2e2109a4ee984f5ba1cb65b950b8d2dc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:23.387535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.398262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e9a78-30ec-4db0-a0c1-2449a2124c4d service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.467377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.468093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.468288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.468747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] PATH_INFO: `/v3/users/33165e00f6474776aae30e5b8597c889` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.469930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33165e00f6474776aae30e5b8597c889)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.471843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.614226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.615612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:23.631041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b902a2-7e7d-4795-b2d6-54483fb5d7d7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.635396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33165e00f6474776aae30e5b8597c889'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.636971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.637671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.637953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.638230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] PATH_INFO: `/v3/projects/c5d1aa9f1b02459f87648af68cc13cdf` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.639447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d1aa9f1b02459f87648af68cc13cdf)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.642153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.698481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33165e00f6474776aae30e5b8597c889'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.712240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33165e00f6474776aae30e5b8597c889'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.713350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33165e00f6474776aae30e5b8597c889'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.726333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33165e00f6474776aae30e5b8597c889'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.732535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 987/1993] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:23 2022] DELETE /identity/v3/users/33165e00f6474776aae30e5b8597c889 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:23.748251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d1aa9f1b02459f87648af68cc13cdf', 'request_id': 'req-acbf6577-2794-4ba8-b543-40129676c573'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.754092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d1aa9f1b02459f87648af68cc13cdf', 'request_id': 'req-acbf6577-2794-4ba8-b543-40129676c573'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.760869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d1aa9f1b02459f87648af68cc13cdf', 'request_id': 'req-acbf6577-2794-4ba8-b543-40129676c573'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.829535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] The token cache is being invalidate because project c5d1aa9f1b02459f87648af68cc13cdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:23.829874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.830634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:23.832228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1007/1994] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:23 2022] DELETE /identity/v3/projects/c5d1aa9f1b02459f87648af68cc13cdf => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:23.836175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.837728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:23.889348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.905536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93e67f-9365-4bda-96a1-67279c0d47d2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.964884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.965984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.965984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.966195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] PATH_INFO: `/v3/users/3c44b046f276448a945ad167ff414232` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.967311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c44b046f276448a945ad167ff414232)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.969263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.123288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.124746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.141456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf6577-2794-4ba8-b543-40129676c573 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.146410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.147196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.147485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.147882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] PATH_INFO: `/v3/projects/37d6daab08404f2db0da883c346d4453` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.149948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d6daab08404f2db0da883c346d4453)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.151854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.156631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c44b046f276448a945ad167ff414232'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.201602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c44b046f276448a945ad167ff414232'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.209154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c44b046f276448a945ad167ff414232'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.209850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c44b046f276448a945ad167ff414232'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.223276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47b73599-8aab-40eb-884d-3d17b1877dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c44b046f276448a945ad167ff414232'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.229731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 988/1995] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:23 2022] DELETE /identity/v3/users/3c44b046f276448a945ad167ff414232 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:24.236830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d6daab08404f2db0da883c346d4453', 'request_id': 'req-0a273c4e-0a5e-4454-9dc4-335269e613d4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.242895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d6daab08404f2db0da883c346d4453', 'request_id': 'req-0a273c4e-0a5e-4454-9dc4-335269e613d4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.249731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d6daab08404f2db0da883c346d4453', 'request_id': 'req-0a273c4e-0a5e-4454-9dc4-335269e613d4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.342774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] The token cache is being invalidate because project 37d6daab08404f2db0da883c346d4453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:24.343135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.343765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a273c4e-0a5e-4454-9dc4-335269e613d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.345320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1008/1996] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:24 2022] DELETE /identity/v3/projects/37d6daab08404f2db0da883c346d4453 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:24.351446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.351901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.352211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.376957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:24.378236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.449439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 989/1997] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:24.450850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.451093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.451370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.476145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:24.477741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.496845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1009/1998] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:24.498780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.509149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a2771-650c-4220-9fa8-c8e597986226 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.514268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.514886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.515166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.515439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.516182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:24.516621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.518345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.528290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.535288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 990/1999] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:24.537049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.549745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efb01b-7b7e-4f3d-90ce-32d670b4092c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.551773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.552455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.552807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.553200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.553842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.556362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.561942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=21,keystone:SELECT=76,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.643949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1010/2000] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:24 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:24.648106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.662513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.663383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229a8fc-8a94-4e64-ae18-fb8f570325c4 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.665604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.729345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.730065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.730418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.730767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.731395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.733397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.740035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df14306-3d47-4f0c-93a6-d8f0499bd3d7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.745167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.745811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.746106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.746373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] PATH_INFO: `/v3/projects/9c5c741be3d542c08ddd8519d0d3e89a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.747507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c5c741be3d542c08ddd8519d0d3e89a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.749461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.801320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 991/2001] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:24 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:24.805316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.814915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3dcc-3ad5-4270-b684-b094ce91de54 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.819514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.820121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.820646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.821059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.821698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.823353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.836968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 992/2002] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:24.838126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.843401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c5c741be3d542c08ddd8519d0d3e89a', 'request_id': 'req-25e8fb0c-be68-4664-a807-721fbe20f671'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.848667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4967e-0fe6-4516-9076-18d201eefc24 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.849063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c5c741be3d542c08ddd8519d0d3e89a', 'request_id': 'req-25e8fb0c-be68-4664-a807-721fbe20f671'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.856602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c5c741be3d542c08ddd8519d0d3e89a', 'request_id': 'req-25e8fb0c-be68-4664-a807-721fbe20f671'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.904077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] The token cache is being invalidate because project 9c5c741be3d542c08ddd8519d0d3e89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:24.904353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.905011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:24.906525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1011/2003] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:24 2022] DELETE /identity/v3/projects/9c5c741be3d542c08ddd8519d0d3e89a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:24.910681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.911992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.913825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.915111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.915388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.915658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.916294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.917978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.924634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8fb0c-be68-4664-a807-721fbe20f671 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.927935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 993/2004] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:24.929836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.930527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.930886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.931162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] PATH_INFO: `/v3/users/a4d36d7fbb8747d28be0babc23f5242c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.932118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.932417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d36d7fbb8747d28be0babc23f5242c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.933783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.934369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.947007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9e76-98be-44e9-96c9-dea6709785b5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.952026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafdf8d-abd2-4c27-b731-c21c54c23d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.952925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafdf8d-abd2-4c27-b731-c21c54c23d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.953089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafdf8d-abd2-4c27-b731-c21c54c23d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.953379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafdf8d-abd2-4c27-b731-c21c54c23d5d admin admin] PATH_INFO: `/v3/projects/8d216c9207c643d6bd472f73724cc3ef/users/34032a0bea6b4ea499b43d7f76381dba/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.969250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafdf8d-abd2-4c27-b731-c21c54c23d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d216c9207c643d6bd472f73724cc3ef, user_id=34032a0bea6b4ea499b43d7f76381dba, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.972021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafdf8d-abd2-4c27-b731-c21c54c23d5d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.098902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 994/2005] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:24 2022] PUT /identity/v3/projects/8d216c9207c643d6bd472f73724cc3ef/users/34032a0bea6b4ea499b43d7f76381dba/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:25.111176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.111467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.111744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.135731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d36d7fbb8747d28be0babc23f5242c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.139849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] MFA Rules not processed for user `34032a0bea6b4ea499b43d7f76381dba`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:25.141094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.143457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d36d7fbb8747d28be0babc23f5242c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.180911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.205764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 995/2006] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:25.214698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d36d7fbb8747d28be0babc23f5242c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.215344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d36d7fbb8747d28be0babc23f5242c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.217847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.219253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.223603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.229194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d36d7fbb8747d28be0babc23f5242c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.235216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1012/2007] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:24 2022] DELETE /identity/v3/users/a4d36d7fbb8747d28be0babc23f5242c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:25.285372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31781fa0-b87a-44b0-b40b-7b683a251653 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.336557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.337247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.337529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.337800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.339730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.400677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.401922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.411683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 996/2008] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:25.436602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.438456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.508499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b952a-1892-47a0-aa2f-d099d297b415 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.513513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.513854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.514106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.514380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] PATH_INFO: `/v3/users/34032a0bea6b4ea499b43d7f76381dba` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.515581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34032a0bea6b4ea499b43d7f76381dba)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.517884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.623822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.626748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.670266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d53ade-f7df-4407-a403-eea6e20cc0da service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.678834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.679600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.681760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.682444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] PATH_INFO: `/v3/projects/5202c563a0c249bfb68885e7f8ad4c63` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.686754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5202c563a0c249bfb68885e7f8ad4c63)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.687225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.694120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34032a0bea6b4ea499b43d7f76381dba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.707264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34032a0bea6b4ea499b43d7f76381dba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.761019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34032a0bea6b4ea499b43d7f76381dba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.761732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34032a0bea6b4ea499b43d7f76381dba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.774977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-672e7a44-54ca-4b28-9d77-b6a823626275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34032a0bea6b4ea499b43d7f76381dba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.780774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1013/2009] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:25 2022] DELETE /identity/v3/users/34032a0bea6b4ea499b43d7f76381dba => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:25.785274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.785563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.785838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.804111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:25.805543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.829974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.832752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5202c563a0c249bfb68885e7f8ad4c63', 'request_id': 'req-47ccacbe-37c5-48c7-9338-acc2fab295d4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.838569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5202c563a0c249bfb68885e7f8ad4c63', 'request_id': 'req-47ccacbe-37c5-48c7-9338-acc2fab295d4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.844894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5202c563a0c249bfb68885e7f8ad4c63', 'request_id': 'req-47ccacbe-37c5-48c7-9338-acc2fab295d4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.871620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1014/2010] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:25.881327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.882782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:25.893701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.943324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f054cfb-41c3-46c7-a1d3-ec7e4357c267 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.981386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] The token cache is being invalidate because project 5202c563a0c249bfb68885e7f8ad4c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:25.981681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.982325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-47ccacbe-37c5-48c7-9338-acc2fab295d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:25.983921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 997/2011] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:25 2022] DELETE /identity/v3/projects/5202c563a0c249bfb68885e7f8ad4c63 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:25.989860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf883f7-5de6-4c39-9314-b53295a092ce None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.990225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf883f7-5de6-4c39-9314-b53295a092ce None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.990574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf883f7-5de6-4c39-9314-b53295a092ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.994569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.999769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.999769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.999769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.999769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.010620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2bf883f7-5de6-4c39-9314-b53295a092ce None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:26.011906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2bf883f7-5de6-4c39-9314-b53295a092ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.064363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.065942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.073355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 998/2012] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:26.080475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1015/2013] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:26.082524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.082812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.083115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.104523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:26.106504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.114497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 999/2014] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:26.118984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.130611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64db007-f88f-481d-a82a-9d713f913d13 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.136440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.137094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.137509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.138384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.139051 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:26.139756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.143102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.162975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1016/2015] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:26.168189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.179342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5ca13-3f7b-4738-b7bd-44ec8f884356 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.184677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.185567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.185912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.186251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.186988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.189232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.300785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1000/2016] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:26.305540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.316073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfed84-c9ee-49ba-b716-5fd84c68794d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.377778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.378391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.378735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.382217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.382217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.382217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.391626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:26.456908 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1017/2017] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:26.462261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.471869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.476133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.477428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f1b24-010d-46a9-8b52-478a20c26b35 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.482910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.483615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.483908 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.484316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.485888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.488347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.501438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1001/2018] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:26.504571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026603e-3170-4655-be2e-23860aa40dcc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.506376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.517929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.518540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9fe22-8fdd-4702-bd0d-30268cf8d7a9 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.518642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.518850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.519127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] PATH_INFO: `/v3/projects/8d216c9207c643d6bd472f73724cc3ef` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.520900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d216c9207c643d6bd472f73724cc3ef)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.522606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.525585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.526281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.526613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.526896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.527540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.529238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.544838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1002/2019] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:26.546072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.557186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2996e-5288-4a91-8314-67c8bb6e4769 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.562111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d357863-3826-4948-8c40-22297adf3ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.562797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d357863-3826-4948-8c40-22297adf3ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.563079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d357863-3826-4948-8c40-22297adf3ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.563351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d357863-3826-4948-8c40-22297adf3ee0 admin admin] PATH_INFO: `/v3/projects/f67b31b5add540418173b931b3efdd49/users/cc943e3d68644e3a81dba6e765186258/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.579267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d357863-3826-4948-8c40-22297adf3ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67b31b5add540418173b931b3efdd49, user_id=cc943e3d68644e3a81dba6e765186258, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.582088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d357863-3826-4948-8c40-22297adf3ee0 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.636303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1003/2020] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:26 2022] PUT /identity/v3/projects/f67b31b5add540418173b931b3efdd49/users/cc943e3d68644e3a81dba6e765186258/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:26.649382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.650390 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.650883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.659679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d216c9207c643d6bd472f73724cc3ef', 'request_id': 'req-c8e154d1-d031-48f5-975c-2450a095ebb2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:26.665771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d216c9207c643d6bd472f73724cc3ef', 'request_id': 'req-c8e154d1-d031-48f5-975c-2450a095ebb2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:26.675099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d216c9207c643d6bd472f73724cc3ef', 'request_id': 'req-c8e154d1-d031-48f5-975c-2450a095ebb2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:26.686248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] MFA Rules not processed for user `cc943e3d68644e3a81dba6e765186258`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:26.687541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.738830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:26.744330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] The token cache is being invalidate because project 8d216c9207c643d6bd472f73724cc3ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:26.744644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:26.745384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c8e154d1-d031-48f5-975c-2450a095ebb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:26.746980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1018/2021] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:26 2022] DELETE /identity/v3/projects/8d216c9207c643d6bd472f73724cc3ef => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:26.755021 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a40b92-bd54-45bf-af2f-287a491ee1ed None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.755283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a40b92-bd54-45bf-af2f-287a491ee1ed None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.755553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a40b92-bd54-45bf-af2f-287a491ee1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.763595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1004/2022] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:26.780337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.780337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.798704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-21a40b92-bd54-45bf-af2f-287a491ee1ed None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:26.800608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-21a40b92-bd54-45bf-af2f-287a491ee1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.878650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d29e8-26f9-4d01-8ae6-708dfec9ca4a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.882757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.883374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.883662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.883937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.887522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1019/2023] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:26.894224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.894961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.900163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.900163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.900364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.907717 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1005/2024] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:26.921220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:26.922380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.937393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1020/2025] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:26.939164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.951831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57baac71-fb16-4653-86de-5b21f17c607f service glance] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.960226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.961017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.961386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.961851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.962520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:26.962833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.964459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.980218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1006/2026] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:26.985538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.996206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e443ab-1834-4907-9140-7a0dfc8e4e65 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.001402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.002420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.002705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.002979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.003706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.005641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:27.112298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1021/2027] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:26 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:27.116270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:27.127507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83575941-3391-43e2-9ea2-bcf606224a6b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.202285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.202853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.203137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.203786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.204132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.206189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:27.282173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1007/2028] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:27 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:27.286506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:27.296949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac045948-4953-486d-9291-864a846a7155 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.301587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.302180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.302521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.302716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.303348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.305228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:27.315264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1022/2029] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:27.319205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:27.330452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f49e49-946a-48e0-9245-ff39ca9a4a82 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.335178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4811b-d4b0-424a-bf86-63826fdd2def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.336402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4811b-d4b0-424a-bf86-63826fdd2def admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.336683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4811b-d4b0-424a-bf86-63826fdd2def admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.337166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4811b-d4b0-424a-bf86-63826fdd2def admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.337820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4811b-d4b0-424a-bf86-63826fdd2def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.339421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4811b-d4b0-424a-bf86-63826fdd2def admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:27.349583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1008/2030] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:27.357245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:27.367414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060acc02-06cd-42e1-8a69-0b1c021d0204 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.372234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.372894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.373205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.373479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] PATH_INFO: `/v3/projects/a979555fb6ee4abbac16b90f40a39f9d/users/58680640515a4562a9b78e05c185151f/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.388104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a979555fb6ee4abbac16b90f40a39f9d, user_id=58680640515a4562a9b78e05c185151f, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.390825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:27.442075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1023/2031] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:27 2022] PUT /identity/v3/projects/a979555fb6ee4abbac16b90f40a39f9d/users/58680640515a4562a9b78e05c185151f/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:27.452969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.453558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.453845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.482539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] MFA Rules not processed for user `58680640515a4562a9b78e05c185151f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:27.483751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:27.509530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:27.550604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1009/2032] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:27.556201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:27.584529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:27.643912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85957-f619-4e2a-8d05-20042c7debf8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.655367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.655862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.656222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.656462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.657127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.659484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:27.774653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1024/2033] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:27 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:27.779685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:27.790612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27155c4f-cf9b-42ba-a308-81763239e375 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:27.855118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:27.855739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.856024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.856306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.857015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:27.858957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.104185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1010/2034] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:27 2022] POST /identity/v3/users => generated 288 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:28.110827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.127607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60204f13-80bf-44b4-adc3-6cc1166c9cf8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.134941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.135409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.135748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.135956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.136794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.138429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.153655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1025/2035] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:28.157863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.170779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f99d4-aa0b-4dd4-a405-8a8b1a5d9fa3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.175281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53172e-69f3-44a0-813a-f6380e134ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.175878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53172e-69f3-44a0-813a-f6380e134ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.176156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53172e-69f3-44a0-813a-f6380e134ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.176475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53172e-69f3-44a0-813a-f6380e134ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.177114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53172e-69f3-44a0-813a-f6380e134ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.178698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53172e-69f3-44a0-813a-f6380e134ef8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.188898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1011/2036] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:28.193443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.205883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd659e5-19ff-422a-a95b-df8565393930 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.214355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.215387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.215675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.216135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] PATH_INFO: `/v3/projects/8d99d99aba2a40e3aae235b51432d53a/users/31ca76886d9a452b85895df0f3ce6227/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.230816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d99d99aba2a40e3aae235b51432d53a, user_id=31ca76886d9a452b85895df0f3ce6227, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.233508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.415042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1026/2037] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:28 2022] PUT /identity/v3/projects/8d99d99aba2a40e3aae235b51432d53a/users/31ca76886d9a452b85895df0f3ce6227/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:28.426972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.427237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.427484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.462902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] MFA Rules not processed for user `31ca76886d9a452b85895df0f3ce6227`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:28.465239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:28.535427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1012/2038] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:28.540443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.633313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6a096-6d6b-4e5f-96ec-6ee72d85702b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.638063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.638662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.638939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.639209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.639877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.642016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.705986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1027/2039] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:28 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:28.709385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.720983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5c40c-f903-462d-afc1-49d4a63c1d3e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.756998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:28.782400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.787112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.787112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.787112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.787112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.787112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.911375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1013/2040] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:28 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:28.915880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.927038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2461f-7554-4398-afa0-f2a02dbf356a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.932132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.932739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.933054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.933333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.933980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.935640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.947935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1028/2041] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:28.952332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.973648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1ddae-6e8b-4ae3-ba51-5312b781329c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.975153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.975819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.976105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.976456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] PATH_INFO: `/v3/projects/3ab07603c29946c2aace2e6e81dcf764/users/ca7fb7c3a74748558ca9886b5c4392df/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.992025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab07603c29946c2aace2e6e81dcf764, user_id=ca7fb7c3a74748558ca9886b5c4392df, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.994837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.173022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1014/2042] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:28 2022] PUT /identity/v3/projects/3ab07603c29946c2aace2e6e81dcf764/users/ca7fb7c3a74748558ca9886b5c4392df/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:29.176821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.193278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a496-a935-4855-8801-75315fd36cbf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.243566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.278243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b34dea-031c-4cf0-93cb-87ad9e3c30f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.279120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b34dea-031c-4cf0-93cb-87ad9e3c30f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.279400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b34dea-031c-4cf0-93cb-87ad9e3c30f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.279872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b34dea-031c-4cf0-93cb-87ad9e3c30f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.280528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b34dea-031c-4cf0-93cb-87ad9e3c30f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.282404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b34dea-031c-4cf0-93cb-87ad9e3c30f6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.301034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1029/2043] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:29.302194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.314804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb6616-f860-4a48-aa29-20184b66d71c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.323942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.324634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.324911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.325209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] PATH_INFO: `/v3/projects/3ab07603c29946c2aace2e6e81dcf764/users/ca7fb7c3a74748558ca9886b5c4392df/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.327522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab07603c29946c2aace2e6e81dcf764, user_id=ca7fb7c3a74748558ca9886b5c4392df, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.330304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.342289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1015/2044] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:29 2022] PUT /identity/v3/projects/3ab07603c29946c2aace2e6e81dcf764/users/ca7fb7c3a74748558ca9886b5c4392df/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:29.353512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.353803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.354082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.383995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] MFA Rules not processed for user `ca7fb7c3a74748558ca9886b5c4392df`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:29.385279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:29.469678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1030/2045] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:29.482212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.484009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:29.543737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682d87c-12d3-4bab-9f90-f6a19e92cfe3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.547553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.548191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.548468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.548743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.607371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.608817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.618853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1016/2046] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:29.724431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.727682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:29.744272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.749730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:29.827514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.830751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9983c-7457-4f5d-b58c-f3a1fdfaa0f8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.836371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.843312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.843312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.843312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] PATH_INFO: `/v3/users/a0247e2771c94cf984ccd89dd935e658` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.843312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0247e2771c94cf984ccd89dd935e658)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.843312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.850518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325c48f-88b4-4bf3-bba8-eaa5c04cdabb service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.856440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.857156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.857502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.857949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] PATH_INFO: `/v3/users/e8b2e816014042a380782d72f9d2ccb2` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.859363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b2e816014042a380782d72f9d2ccb2)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.861228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:30.066521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0247e2771c94cf984ccd89dd935e658'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.073771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0247e2771c94cf984ccd89dd935e658'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.076466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:30.090183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b2e816014042a380782d72f9d2ccb2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.139127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0247e2771c94cf984ccd89dd935e658'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.139777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0247e2771c94cf984ccd89dd935e658'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.152948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4c4a5235-2602-4f5e-bfb4-c9b7e8c7655c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0247e2771c94cf984ccd89dd935e658'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.159208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1031/2047] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:29 2022] DELETE /identity/v3/users/a0247e2771c94cf984ccd89dd935e658 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:30.164114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:30.173346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:30.173346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:30.183492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:30.184684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:30.201391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b2e816014042a380782d72f9d2ccb2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.209267 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b2e816014042a380782d72f9d2ccb2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.209925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b2e816014042a380782d72f9d2ccb2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.233711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-464ee96e-4be3-4cef-9668-fc25a5121523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b2e816014042a380782d72f9d2ccb2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:30.240537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1017/2048] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:29 2022] DELETE /identity/v3/users/e8b2e816014042a380782d72f9d2ccb2 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:30.244541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1032/2049] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:30.245433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:30.245726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:30.246005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:30.254630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:30.261429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:30.265898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:30.267124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:30.324506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8da8c-b6b2-4ac3-b386-51aeff19a6cf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:30.333799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1018/2050] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:30.344154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:30.401049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:30.404679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:30.404679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:30.404679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:30.418414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:30.419901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:30.432240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1033/2051] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:30.433430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab5fff-f28b-4695-9fed-f8a18594ab2d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:30.438907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:30.439535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:30.440087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:30.440399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:30.453910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:30.455477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:30.467951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1019/2052] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:31.152171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.153818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:31.159105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.161177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:31.176953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:31.177853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136e8e-c60c-4c9c-9795-882930966ea1 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:31.181955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b54fc-5e1b-4f68-839a-73cb0f4665cb service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:31.183936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:31.184206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.184493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.184770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] PATH_INFO: `/v3/projects/ae61ce475940488883ddf66b78edb2cf` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.185954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae61ce475940488883ddf66b78edb2cf)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:31.187882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:31.188439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:31.189185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.189574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.189915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] PATH_INFO: `/v3/projects/6cbae98fb64443d8b14f7a45d1a659cc` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.191182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cbae98fb64443d8b14f7a45d1a659cc)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:31.193343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:31.304354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:31.329031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae61ce475940488883ddf66b78edb2cf', 'request_id': 'req-42bc31d1-21bd-438c-85c2-76eb837f6409'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.333859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cbae98fb64443d8b14f7a45d1a659cc', 'request_id': 'req-e852d258-29e4-4c86-87cb-96d81904dde3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.337925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae61ce475940488883ddf66b78edb2cf', 'request_id': 'req-42bc31d1-21bd-438c-85c2-76eb837f6409'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.342939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cbae98fb64443d8b14f7a45d1a659cc', 'request_id': 'req-e852d258-29e4-4c86-87cb-96d81904dde3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.350813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae61ce475940488883ddf66b78edb2cf', 'request_id': 'req-42bc31d1-21bd-438c-85c2-76eb837f6409'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.350950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cbae98fb64443d8b14f7a45d1a659cc', 'request_id': 'req-e852d258-29e4-4c86-87cb-96d81904dde3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.424879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] The token cache is being invalidate because project ae61ce475940488883ddf66b78edb2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:31.433058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.433058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.433058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1034/2053] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:31 2022] DELETE /identity/v3/projects/ae61ce475940488883ddf66b78edb2cf => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:31.433058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.434446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:31.494759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] The token cache is being invalidate because project 6cbae98fb64443d8b14f7a45d1a659cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:31.495333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.495724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e852d258-29e4-4c86-87cb-96d81904dde3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.497565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1020/2054] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:31 2022] DELETE /identity/v3/projects/6cbae98fb64443d8b14f7a45d1a659cc => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:31.501942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efccca-7a1f-426a-8482-3b30d2f19d70 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.502484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efccca-7a1f-426a-8482-3b30d2f19d70 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.502770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efccca-7a1f-426a-8482-3b30d2f19d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.519965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc31d1-21bd-438c-85c2-76eb837f6409 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:31.526538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-24efccca-7a1f-426a-8482-3b30d2f19d70 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:31.527823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-24efccca-7a1f-426a-8482-3b30d2f19d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:31.611157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:31.611812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.612092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.612367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] PATH_INFO: `/v3/users/25c6c6881a0d45d1b48bfd8eaea6c18c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.613543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c6c6881a0d45d1b48bfd8eaea6c18c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:31.615636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:31.621121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1021/2055] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:31.637881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.637881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.637881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.663356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:31.665175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:31.674187 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1022/2056] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:31.678440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.696235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff3afd-3e8e-4a51-b0d8-8bbe360f9b7e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:31.699636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:31.700492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.700780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.701083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.701753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:31.702056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:31.703701 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:31.721179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1023/2057] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:31.724616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.736174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00224-f437-4a9d-8244-7a285c0f2eae admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:31.741280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:31.741984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:31.742275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:31.742751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:31.743363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:31.745299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:31.781087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c6c6881a0d45d1b48bfd8eaea6c18c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.789026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c6c6881a0d45d1b48bfd8eaea6c18c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.830016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1024/2058] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:31 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:31.830702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.833807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:31.854336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c6c6881a0d45d1b48bfd8eaea6c18c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.856216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c6c6881a0d45d1b48bfd8eaea6c18c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.872832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:31.911906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:31.912415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c6c6881a0d45d1b48bfd8eaea6c18c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:31.919954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0ba75-16f1-4f77-965d-c87924f64e92 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:31.920872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1035/2059] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:31 2022] DELETE /identity/v3/users/25c6c6881a0d45d1b48bfd8eaea6c18c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:31.921687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:31.923191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:32.021898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.022545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.022829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.023753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.030095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:32.081494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca243a1-a1bc-48fd-bb45-bc0c18f66b59 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.093793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.094431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.094824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.095807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.096743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.100791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.133134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.134828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.151180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1025/2060] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:32.231120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1036/2061] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:31 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:32.235332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.247766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edbc66-8379-48f7-b99d-62c1676f07dc service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.252751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.253640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.254074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.254519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.255505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.257603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.269200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1026/2062] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:32.273734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.284048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413379cc-4889-400f-b9d1-31d578bad61d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.289446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11b264-3bb2-4a12-9cd2-232a3a300bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.292018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11b264-3bb2-4a12-9cd2-232a3a300bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.292018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11b264-3bb2-4a12-9cd2-232a3a300bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.292018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11b264-3bb2-4a12-9cd2-232a3a300bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.292018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11b264-3bb2-4a12-9cd2-232a3a300bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.293150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11b264-3bb2-4a12-9cd2-232a3a300bd1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.303453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1037/2063] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:32.310987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.326467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f42c13-f0d0-42b7-b436-695f8478b455 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.332334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.333003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.333298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.333600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] PATH_INFO: `/v3/projects/e43f6ad26c6a4dcdb27c9d9ba706cd0e/users/d4e91c327d0c4b5dbae54cb915aca3e6/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.352144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43f6ad26c6a4dcdb27c9d9ba706cd0e, user_id=d4e91c327d0c4b5dbae54cb915aca3e6, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.355962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.436073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1027/2064] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:32 2022] PUT /identity/v3/projects/e43f6ad26c6a4dcdb27c9d9ba706cd0e/users/d4e91c327d0c4b5dbae54cb915aca3e6/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:32.441204 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.441499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.446914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.476777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:32.476777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:32.553017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1038/2065] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:32.564165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.569644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:32.633378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b82ca-63ef-418b-a3c7-093e8b9f6295 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.638861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72f96-a769-45d8-93af-643f9b67ff77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.638861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72f96-a769-45d8-93af-643f9b67ff77 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.638861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72f96-a769-45d8-93af-643f9b67ff77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.638861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72f96-a769-45d8-93af-643f9b67ff77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.649260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c72f96-a769-45d8-93af-643f9b67ff77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.650998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c72f96-a769-45d8-93af-643f9b67ff77 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.661650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1028/2066] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:32.724308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.725877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:32.739438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca3387-c2d3-42f9-af0e-9c795735f3c7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.744505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.745254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.745548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.745810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] PATH_INFO: `/v3/projects/122c3470c77643f09b0243ac6563a296` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.746927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=122c3470c77643f09b0243ac6563a296)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.748833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.867685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122c3470c77643f09b0243ac6563a296', 'request_id': 'req-ca989a04-9a2f-40e4-b353-0b68427b83a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:32.873394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122c3470c77643f09b0243ac6563a296', 'request_id': 'req-ca989a04-9a2f-40e4-b353-0b68427b83a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:32.874586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:32.880688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122c3470c77643f09b0243ac6563a296', 'request_id': 'req-ca989a04-9a2f-40e4-b353-0b68427b83a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:32.975683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] The token cache is being invalidate because project 122c3470c77643f09b0243ac6563a296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:32.975995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:32.977044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ca989a04-9a2f-40e4-b353-0b68427b83a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:32.979721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1039/2067] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:32 2022] DELETE /identity/v3/projects/122c3470c77643f09b0243ac6563a296 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:32.988454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.988819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.989145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.019897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:33.021237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:33.056936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:33.107820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1029/2068] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:33.115732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.115934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.116221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.136165 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:33.137414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:33.147702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1040/2069] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:33.153422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.165119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b932d36-0af0-495d-83ae-f19513fe8e7d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.170283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.170899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.171814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.172114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.173122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:33.174015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:33.175729 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:33.191617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1030/2070] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:33.192859 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.202841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075348c-8bdd-4ecf-b287-3eb8b470773d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.208172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.208362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.208644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.209133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.209768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:33.212708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:33.379568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1041/2071] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:33 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:33.380703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.391828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9c12-0cbe-4c6a-ae4d-d0548b481e39 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.450413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.451153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.451430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.451704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.452400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:33.455268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:33.575067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1031/2072] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:33 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:33.579311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.590589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eafc8-57cf-4f81-acd1-d6886de04c3d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.595298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.595906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.596188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.596568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.597263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:33.598851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:33.611206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1042/2073] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:33.615505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.636615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b53e-3427-43d6-b91e-50c55661d739 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.642424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01015914-d9dd-4fcb-91c7-5806a814dafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.643044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015914-d9dd-4fcb-91c7-5806a814dafc admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.643329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015914-d9dd-4fcb-91c7-5806a814dafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.643603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015914-d9dd-4fcb-91c7-5806a814dafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.644283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01015914-d9dd-4fcb-91c7-5806a814dafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:33.646237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01015914-d9dd-4fcb-91c7-5806a814dafc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:33.670662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1032/2074] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:33.674994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.685741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5697a-33e8-42a9-b62e-f36502dc9833 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.690297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.690952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.691239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.691643 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] PATH_INFO: `/v3/projects/c5fffc0ef92a4250ae4743af647dda32/users/ac7fb1e6100046ea900c66b4e4f19674/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.707844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fffc0ef92a4250ae4743af647dda32, user_id=ac7fb1e6100046ea900c66b4e4f19674, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:33.710603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:33.820569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1043/2075] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:33 2022] PUT /identity/v3/projects/c5fffc0ef92a4250ae4743af647dda32/users/ac7fb1e6100046ea900c66b4e4f19674/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:33.845427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.845724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.846004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.875855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] MFA Rules not processed for user `ac7fb1e6100046ea900c66b4e4f19674`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:33.877106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:33.901745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.903455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:33.953505 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1033/2076] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:33.958067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:33.979199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc2a28-4f5d-4690-80d3-5eeb973214c4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:33.983601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:33.984140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:33.984760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:33.985103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:33.985380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:33.988409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:34.033645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0490d2-54c9-440b-91f8-22a185e2faa9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.043125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.045214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.045214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.046301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.048032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.050834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.072181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.073690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378656cc-51b0-42b1-82e9-2be30ae85b1c service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.083813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1044/2077] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:34.153838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.158067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.158674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.176367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1034/2078] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:33 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:34.180920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.202783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] MFA Rules not processed for user `d4e91c327d0c4b5dbae54cb915aca3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:34.204074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:34.229995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:34.301066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1045/2079] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:34.302416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0574c-c14a-4d5b-b263-a89e2c468695 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.308487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.309212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.309496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.309861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.313110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.313110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.317071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.317071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:34.392746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b472d68-0b78-4ee2-aa21-c6c03f663778 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.402980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.402980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.402980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.402980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.414243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.415817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.426364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1035/2080] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:34 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:34.429712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1046/2081] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:34.430689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.442089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce2017-fb43-45d1-af48-94941763cfaf admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.455128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.455977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.456832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.456832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.458289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.460115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.475272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1036/2082] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:34.479265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.493391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445bb1c-d8d8-4c68-9fec-065f94912833 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.498748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.499411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.499689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.499962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] PATH_INFO: `/v3/projects/7fbb0768cead4813bfe8b5f74a9f5efe/users/28b8cf4de84445e2807480f9371b751f/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.523484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fbb0768cead4813bfe8b5f74a9f5efe, user_id=28b8cf4de84445e2807480f9371b751f, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.526412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.602928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1047/2083] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:34 2022] PUT /identity/v3/projects/7fbb0768cead4813bfe8b5f74a9f5efe/users/28b8cf4de84445e2807480f9371b751f/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:34.607118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.619837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc870d2f-d750-4af5-9ee0-6ccbbd1a1ecd admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.698020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92bc5f-9e82-4c5d-ba8e-32620f7b1798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.698635 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92bc5f-9e82-4c5d-ba8e-32620f7b1798 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.698917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92bc5f-9e82-4c5d-ba8e-32620f7b1798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.699202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92bc5f-9e82-4c5d-ba8e-32620f7b1798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.699876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92bc5f-9e82-4c5d-ba8e-32620f7b1798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.702333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92bc5f-9e82-4c5d-ba8e-32620f7b1798 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.713855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1037/2084] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:34.718082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.729206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea423b-3cdc-463f-9210-cdde903bb683 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.733887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.741988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.741988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.741988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] PATH_INFO: `/v3/projects/7fbb0768cead4813bfe8b5f74a9f5efe/users/28b8cf4de84445e2807480f9371b751f/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.741988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fbb0768cead4813bfe8b5f74a9f5efe, user_id=28b8cf4de84445e2807480f9371b751f, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.741988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.750418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1048/2085] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:34 2022] PUT /identity/v3/projects/7fbb0768cead4813bfe8b5f74a9f5efe/users/28b8cf4de84445e2807480f9371b751f/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:34.760845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2be16a-3ca4-45b6-bbcc-3795f7e268fe None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.761162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2be16a-3ca4-45b6-bbcc-3795f7e268fe None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.761486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2be16a-3ca4-45b6-bbcc-3795f7e268fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.793730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-eb2be16a-3ca4-45b6-bbcc-3795f7e268fe None None] MFA Rules not processed for user `28b8cf4de84445e2807480f9371b751f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:34.795021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-eb2be16a-3ca4-45b6-bbcc-3795f7e268fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:34.803965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:34.865050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1038/2086] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:34.910668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.971071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50d0eb-d3ce-4eff-87f1-61abc94b10ad admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.974767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.975377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.975658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.975931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.986040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.987535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.997629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1049/2087] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:36.258723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e933e-cc76-4563-8edc-ac40ffba6da6 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:36.261125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e933e-cc76-4563-8edc-ac40ffba6da6 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:36.261831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e933e-cc76-4563-8edc-ac40ffba6da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:36.285629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:36.290423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ca7e933e-cc76-4563-8edc-ac40ffba6da6 None None] MFA Rules not processed for user `d4e91c327d0c4b5dbae54cb915aca3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:36.292150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ca7e933e-cc76-4563-8edc-ac40ffba6da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:36.349405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:36.374514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1039/2088] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:36.388751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab547d0-b119-4836-be6f-c94b54292888 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:36.393443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:36.394089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:36.394585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:36.395036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] PATH_INFO: `/v3/users/ac7fb1e6100046ea900c66b4e4f19674` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:36.398800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7fb1e6100046ea900c66b4e4f19674)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:36.398800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:36.596015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7fb1e6100046ea900c66b4e4f19674'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:36.609692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7fb1e6100046ea900c66b4e4f19674'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:36.681411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7fb1e6100046ea900c66b4e4f19674'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:36.682422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7fb1e6100046ea900c66b4e4f19674'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:36.713195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7fb1e6100046ea900c66b4e4f19674'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:36.725311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:36.725621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:36.725897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:36.729229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1050/2089] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:36 2022] DELETE /identity/v3/users/ac7fb1e6100046ea900c66b4e4f19674 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:36.748008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:36.749314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:36.841677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1040/2090] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:36.856501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:36.859004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:36.945776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e346c-c0c3-426f-b271-9f185c894160 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:36.950572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:36.951216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:36.951487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:36.951756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:36.963841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:36.965353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:36.983210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1051/2091] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:37.497622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:37.499589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:37.514274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471cca7-e165-4844-8ab5-b4d6f3638094 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:37.519052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:37.519872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.520147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.520445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] PATH_INFO: `/v3/projects/c5fffc0ef92a4250ae4743af647dda32` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.521783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fffc0ef92a4250ae4743af647dda32)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:37.524407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:37.645358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:37.649784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fffc0ef92a4250ae4743af647dda32', 'request_id': 'req-ea414b59-88b1-43e6-bb90-cf39161d9eae'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:37.659608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fffc0ef92a4250ae4743af647dda32', 'request_id': 'req-ea414b59-88b1-43e6-bb90-cf39161d9eae'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:37.669138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fffc0ef92a4250ae4743af647dda32', 'request_id': 'req-ea414b59-88b1-43e6-bb90-cf39161d9eae'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:37.773479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] The token cache is being invalidate because project c5fffc0ef92a4250ae4743af647dda32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:37.773994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:37.775530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:37.785353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1041/2092] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:37 2022] DELETE /identity/v3/projects/c5fffc0ef92a4250ae4743af647dda32 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:37.786336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:37.799625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159320b-3774-4ba4-b62a-f5fee29b50e7 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:37.872312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:37.886865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:37.887429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.887924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.888456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] PATH_INFO: `/v3/users/28b8cf4de84445e2807480f9371b751f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.893701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b8cf4de84445e2807480f9371b751f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:37.907267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:38.105616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b8cf4de84445e2807480f9371b751f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:38.112978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b8cf4de84445e2807480f9371b751f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:38.195852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b8cf4de84445e2807480f9371b751f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:38.196538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b8cf4de84445e2807480f9371b751f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:38.222288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b8cf4de84445e2807480f9371b751f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:38.233597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1052/2093] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:37 2022] DELETE /identity/v3/users/28b8cf4de84445e2807480f9371b751f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:38.866335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:38.868722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:38.894542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea414b59-88b1-43e6-bb90-cf39161d9eae admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:38.967220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:38.967884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:38.968172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:38.968618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] PATH_INFO: `/v3/projects/7fbb0768cead4813bfe8b5f74a9f5efe` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:38.969829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fbb0768cead4813bfe8b5f74a9f5efe)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:38.972472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.073777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.075247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:39.092593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76444-30cd-42da-a1bb-ef4dfd1bb3c5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.103011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fbb0768cead4813bfe8b5f74a9f5efe', 'request_id': 'req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.110098 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fbb0768cead4813bfe8b5f74a9f5efe', 'request_id': 'req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.116911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fbb0768cead4813bfe8b5f74a9f5efe', 'request_id': 'req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.164365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.165127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.165483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.167579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] PATH_INFO: `/v3/users/58680640515a4562a9b78e05c185151f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.167579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=58680640515a4562a9b78e05c185151f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.169045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.186052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] The token cache is being invalidate because project 7fbb0768cead4813bfe8b5f74a9f5efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:39.186342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.186990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b39ddfbc-ff71-495f-99b5-2e5f8604de9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.190314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1042/2094] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:38 2022] DELETE /identity/v3/projects/7fbb0768cead4813bfe8b5f74a9f5efe => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:39.195397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ccd84-8e9d-4b44-881c-75223474b669 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.195887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ccd84-8e9d-4b44-881c-75223474b669 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.196230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ccd84-8e9d-4b44-881c-75223474b669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.217086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-926ccd84-8e9d-4b44-881c-75223474b669 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:39.218300 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-926ccd84-8e9d-4b44-881c-75223474b669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:39.226321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1043/2095] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:39.234032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.236455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.236744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.256709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:39.258147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:39.269787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1044/2096] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:39.273182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.289225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a618e-4502-417a-98a5-cf72acbf3088 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.295519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58680640515a4562a9b78e05c185151f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.296092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.296709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.297018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.297357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.298014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:39.298324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.300111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.305571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58680640515a4562a9b78e05c185151f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.313486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1045/2097] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:39.318108 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.330629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeeebb-93d3-4abe-98d0-b7ee39ba7c7e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.335617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.336229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.336658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.336939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.337599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.339523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.379017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58680640515a4562a9b78e05c185151f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.379845 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58680640515a4562a9b78e05c185151f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.415543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:39.428956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-78b16434-05f1-4769-a8c1-0be6c2f050ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58680640515a4562a9b78e05c185151f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:39.439912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1053/2098] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:39 2022] DELETE /identity/v3/users/58680640515a4562a9b78e05c185151f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:39.444214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.445388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.445820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.468811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1046/2099] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:39 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:39.476797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.478742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:39.481140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:39.482862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:39.559582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:39.578715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1054/2100] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:39.590349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.596325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:39.694473 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e04247-af3b-4da7-8fda-8027b373dbaa admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.703227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.705792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.705792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.705792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.705792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.710462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.764185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad0fda-8cc6-4feb-a508-52600331b16d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.769027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.769027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.780442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.780733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.807145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.809264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.824896 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1055/2101] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:39.848615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1047/2102] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:39 2022] POST /identity/v3/users => generated 293 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:39.849391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.862328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509dbe6-952b-4211-a371-4b571d9039e0 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.870455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.871197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.871514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.871784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.872432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.874604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.895244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1056/2103] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:39.896341 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.916881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd85292-61a9-484e-97fd-375844ac044a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.922606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68441ea5-7083-4143-92a8-6ef318fb61f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.923798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441ea5-7083-4143-92a8-6ef318fb61f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.924210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441ea5-7083-4143-92a8-6ef318fb61f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.924551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441ea5-7083-4143-92a8-6ef318fb61f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:39.925927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68441ea5-7083-4143-92a8-6ef318fb61f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:39.927877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68441ea5-7083-4143-92a8-6ef318fb61f4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:39.940615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1048/2104] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:39.945248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:39.959940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93d7ba-d04b-453e-ba1a-d339ed5efbb9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:39.977912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:39.978744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:39.979093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:39.979438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] PATH_INFO: `/v3/projects/51ffd8208b694c3e81ce7ffa152596f1/users/9c6344c992cb4e408732b94b8eb5e4ad/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.007882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ffd8208b694c3e81ce7ffa152596f1, user_id=9c6344c992cb4e408732b94b8eb5e4ad, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.010877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.151397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1057/2105] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:39 2022] PUT /identity/v3/projects/51ffd8208b694c3e81ce7ffa152596f1/users/9c6344c992cb4e408732b94b8eb5e4ad/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:40.152812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.153143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.153431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.220046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] MFA Rules not processed for user `9c6344c992cb4e408732b94b8eb5e4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:40.221514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:40.308875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1049/2106] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:40.320474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.324573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:40.395976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea919c-7cd8-4133-a701-f547ee4af41d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.400292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.400941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.401193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.401469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.411071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.412642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.422274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1058/2107] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:40.444102 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.446693 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:40.626936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8671b1-d9ac-4a9b-9102-2b61c27f630c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.640786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.641495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.641783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.642064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] PATH_INFO: `/v3/projects/a979555fb6ee4abbac16b90f40a39f9d` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.645508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=a979555fb6ee4abbac16b90f40a39f9d)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.648002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.798742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a979555fb6ee4abbac16b90f40a39f9d', 'request_id': 'req-025d0f13-27dd-41b8-bb2c-aad2f2556ded'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:40.807069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a979555fb6ee4abbac16b90f40a39f9d', 'request_id': 'req-025d0f13-27dd-41b8-bb2c-aad2f2556ded'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:40.815599 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a979555fb6ee4abbac16b90f40a39f9d', 'request_id': 'req-025d0f13-27dd-41b8-bb2c-aad2f2556ded'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:40.888139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] The token cache is being invalidate because project a979555fb6ee4abbac16b90f40a39f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:40.888493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:40.889289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-025d0f13-27dd-41b8-bb2c-aad2f2556ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:40.893804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1050/2108] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:40 2022] DELETE /identity/v3/projects/a979555fb6ee4abbac16b90f40a39f9d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:40.896579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.914387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b397b-7090-458f-a421-7aa6200dbdcd service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.981978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.985783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.985783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.985783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] PATH_INFO: `/v3/users/31ca76886d9a452b85895df0f3ce6227` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.985783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ca76886d9a452b85895df0f3ce6227)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.986760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:41.262679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:41.267177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ca76886d9a452b85895df0f3ce6227'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:41.276877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ca76886d9a452b85895df0f3ce6227'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:41.338945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ca76886d9a452b85895df0f3ce6227'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:41.340037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ca76886d9a452b85895df0f3ce6227'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:41.395225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d05c2f66-fc0e-48e9-8cfb-89329b7d96bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ca76886d9a452b85895df0f3ce6227'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:41.404377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1059/2109] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:40 2022] DELETE /identity/v3/users/31ca76886d9a452b85895df0f3ce6227 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:41.441745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:41.497698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32173a38-01f0-4e30-bcbd-8a9f3b8f86b9 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:41.497894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32173a38-01f0-4e30-bcbd-8a9f3b8f86b9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:41.498369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32173a38-01f0-4e30-bcbd-8a9f3b8f86b9 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:41.500028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1051/2110] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:41.533487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:41.534388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:41.534779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:41.557853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] MFA Rules not processed for user `8c82e23142a343b096a493e01071a506`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:41.559480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:41.667968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1060/2111] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:26:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:41.673440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32173a38-01f0-4e30-bcbd-8a9f3b8f86b9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:41.675813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-32173a38-01f0-4e30-bcbd-8a9f3b8f86b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:41.687962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:41.692480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:41.730478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c4e1-048a-463b-af4d-decd8e8877f0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:41.735460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:41.749324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:41.749324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:41.749324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:41.758645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:41.760725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:41.781833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1061/2112] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:41.801135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32173a38-01f0-4e30-bcbd-8a9f3b8f86b9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:41.806956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:41.807605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:41.807944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:41.808227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] PATH_INFO: `/v3/users/a46db4eba75f4c49a394bffe4f220fe0` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:41.809950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a46db4eba75f4c49a394bffe4f220fe0)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:41.812080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:42.107699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a46db4eba75f4c49a394bffe4f220fe0'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.190965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a46db4eba75f4c49a394bffe4f220fe0'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.197960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:42.199419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:42.199969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a46db4eba75f4c49a394bffe4f220fe0'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.201476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a46db4eba75f4c49a394bffe4f220fe0'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.217715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a46db4eba75f4c49a394bffe4f220fe0'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.224736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1052/2113] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:41 2022] DELETE /identity/v3/users/a46db4eba75f4c49a394bffe4f220fe0 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:42.228178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae5caf-1582-48dd-955f-ddbc17924fed service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:42.299279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:42.299981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:42.300269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:42.300548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] PATH_INFO: `/v3/projects/8d99d99aba2a40e3aae235b51432d53a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:42.302494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d99d99aba2a40e3aae235b51432d53a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:42.305462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:42.438176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d99d99aba2a40e3aae235b51432d53a', 'request_id': 'req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.443874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d99d99aba2a40e3aae235b51432d53a', 'request_id': 'req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.451446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d99d99aba2a40e3aae235b51432d53a', 'request_id': 'req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.518273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:42.520133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:42.547670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc602ea-9d5a-45aa-bdf6-df92ae3f72c4 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:42.562736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] The token cache is being invalidate because project 8d99d99aba2a40e3aae235b51432d53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:42.563637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.563803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.565687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1062/2114] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:42 2022] DELETE /identity/v3/projects/8d99d99aba2a40e3aae235b51432d53a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:42.569947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:42.571575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:42.617820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:42.618466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:42.618752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:42.619032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] PATH_INFO: `/v3/projects/e52ecd30866b4733acd28c43adbc29a0` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:42.620206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52ecd30866b4733acd28c43adbc29a0)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:42.622171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:42.649049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff5ff8-67fd-4b95-abf7-5b2bd995e644 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:42.654111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:42.654933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:42.655406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:42.655946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] PATH_INFO: `/v3/users/ca7fb7c3a74748558ca9886b5c4392df` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:42.657422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca7fb7c3a74748558ca9886b5c4392df)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:42.659625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:42.730309 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52ecd30866b4733acd28c43adbc29a0', 'request_id': 'req-bea03679-bd81-4b58-996a-bb5201e05797'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.739118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52ecd30866b4733acd28c43adbc29a0', 'request_id': 'req-bea03679-bd81-4b58-996a-bb5201e05797'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.746442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52ecd30866b4733acd28c43adbc29a0', 'request_id': 'req-bea03679-bd81-4b58-996a-bb5201e05797'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.812569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] The token cache is being invalidate because project e52ecd30866b4733acd28c43adbc29a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:42.812857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.813538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-bea03679-bd81-4b58-996a-bb5201e05797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.815244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1053/2115] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:42 2022] DELETE /identity/v3/projects/e52ecd30866b4733acd28c43adbc29a0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:42.820734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeccad-9bb8-442a-a256-35cc68b96064 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:42.821281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeccad-9bb8-442a-a256-35cc68b96064 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:42.821636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeccad-9bb8-442a-a256-35cc68b96064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:42.823484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca7fb7c3a74748558ca9886b5c4392df'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.830502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca7fb7c3a74748558ca9886b5c4392df'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.841686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8bbeccad-9bb8-442a-a256-35cc68b96064 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:42.842900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8bbeccad-9bb8-442a-a256-35cc68b96064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:42.875822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:42.904253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1054/2116] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:42.912022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:42.912316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:42.912598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:42.932722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:42.932878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca7fb7c3a74748558ca9886b5c4392df'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.933596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca7fb7c3a74748558ca9886b5c4392df'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.933978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:42.942036 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1055/2117] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:42.945032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:42.946588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:42.948600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca7fb7c3a74748558ca9886b5c4392df'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:42.955296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1063/2118] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:42 2022] DELETE /identity/v3/users/ca7fb7c3a74748558ca9886b5c4392df => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:42.956032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df764e9-33d1-4f71-9ff6-fae5faba9d5a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.015638 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.016281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.016563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.016841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.017562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:43.017872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.019500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.035753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1056/2119] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:43.037146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.048048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddb2e6-f4c7-48ce-8f40-a30db9e0e490 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.052715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.053407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.053688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.054049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.054683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.056965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.149135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1064/2120] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:43 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:43.159379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.171503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6924-30c5-4d3c-9f42-9a46c24d4999 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.240079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.240706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.241012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.241314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.242018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.243986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.333097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1057/2121] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:43 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:43.337386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.349370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33caecc-8674-4bdd-b006-045a54d4fb4c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.354442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.355176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.355365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.355642 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.356302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.357987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.373035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1065/2122] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:43.374756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.387862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903990b-bb64-422f-90ab-2e6139b985ce admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.393359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e143e0-ffbd-45f8-831d-3733bd337d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.394211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e143e0-ffbd-45f8-831d-3733bd337d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.394659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e143e0-ffbd-45f8-831d-3733bd337d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.395195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e143e0-ffbd-45f8-831d-3733bd337d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.395861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e143e0-ffbd-45f8-831d-3733bd337d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.398478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e143e0-ffbd-45f8-831d-3733bd337d1f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.414958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1058/2123] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:43.416098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.426945 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ae85d-a67c-4891-932f-0897406b80dd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.434014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.439301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.439301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.439301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] PATH_INFO: `/v3/projects/a60a5752fcba46359ef5275f506c933a/users/1945fed03f73472db818c4e5ca015eb7/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.451594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60a5752fcba46359ef5275f506c933a, user_id=1945fed03f73472db818c4e5ca015eb7, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.454323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.556424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1066/2124] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:43 2022] PUT /identity/v3/projects/a60a5752fcba46359ef5275f506c933a/users/1945fed03f73472db818c4e5ca015eb7/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:43.561515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.562023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.562115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.581296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:43.582532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:43.643219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1059/2125] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:43.653660 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.655452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:43.690926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:43.722302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175015-b1ff-4f89-a8f4-e995cfa8ee9e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.726906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89d59f-d70d-45df-8c1d-caf39253ed13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.727849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89d59f-d70d-45df-8c1d-caf39253ed13 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.727849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89d59f-d70d-45df-8c1d-caf39253ed13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.728084 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89d59f-d70d-45df-8c1d-caf39253ed13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.737916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89d59f-d70d-45df-8c1d-caf39253ed13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.738711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:43.739687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89d59f-d70d-45df-8c1d-caf39253ed13 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.750449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1067/2126] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:43.910656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.912172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:43.927320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66877d-03fb-4066-af46-ce3e1aac6e70 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.933694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.936286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.936286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.936286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] PATH_INFO: `/v3/projects/3ab07603c29946c2aace2e6e81dcf764` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.937680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab07603c29946c2aace2e6e81dcf764)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.940428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.058830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab07603c29946c2aace2e6e81dcf764', 'request_id': 'req-7742802d-7180-41fc-8228-9a42b7f001ba'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:44.064040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab07603c29946c2aace2e6e81dcf764', 'request_id': 'req-7742802d-7180-41fc-8228-9a42b7f001ba'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:44.072594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab07603c29946c2aace2e6e81dcf764', 'request_id': 'req-7742802d-7180-41fc-8228-9a42b7f001ba'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:44.167304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] The token cache is being invalidate because project 3ab07603c29946c2aace2e6e81dcf764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:44.172132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:44.172132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-7742802d-7180-41fc-8228-9a42b7f001ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:44.172132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1060/2127] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:43 2022] DELETE /identity/v3/projects/3ab07603c29946c2aace2e6e81dcf764 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:44.174808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.175325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.175669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.196127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:44.197708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:44.256022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1068/2128] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:44.264452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.264768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.265063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.285830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:44.292648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:44.296153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1061/2129] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:44.300025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:44.311049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf3369-f2c2-4d0c-b565-e4b093ecfbba None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:44.323012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:44.323688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.323950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.324229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.324932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:44.325816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:44.327984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.347402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1069/2130] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:44.349292 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:44.359899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bcc65-82bc-4505-910d-bf8a16e838db None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:44.364814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:44.365494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.365770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.366043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.367410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:44.369494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.506357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1062/2131] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:44 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:44.511159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:44.521944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811778-69d7-40bc-b489-98bf5d994dac admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:44.528893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:44.600929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:44.601983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.602418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.602877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.603666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:44.606022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.670538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:44.736411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1070/2132] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:44 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:44.740342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:44.752144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd3f10-c777-499b-b595-3835130e3969 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:44.757955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:44.759539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.760004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.760569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.765033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:44.765033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.774161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1063/2133] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:44.778373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:44.790382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6b94-71e1-4182-810f-c466cacbfea5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:44.796000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7243a-9834-455a-b600-1620ac7bcb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:44.796902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7243a-9834-455a-b600-1620ac7bcb12 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.797267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7243a-9834-455a-b600-1620ac7bcb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.797577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7243a-9834-455a-b600-1620ac7bcb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.798429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7243a-9834-455a-b600-1620ac7bcb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:44.800323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7243a-9834-455a-b600-1620ac7bcb12 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.813872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1071/2134] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:44.818774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:44.829652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37da6-ec2b-4d45-9e35-386ccebaf8f4 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:44.834814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:44.835705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.836232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.836624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] PATH_INFO: `/v3/projects/40e87f1f33e64a789cb7c8c7d5779d95/users/e84ba8e4aac24fd78dac290b2c9f1c57/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.853399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e87f1f33e64a789cb7c8c7d5779d95, user_id=e84ba8e4aac24fd78dac290b2c9f1c57, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:44.856241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:44.925610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1064/2135] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:44 2022] PUT /identity/v3/projects/40e87f1f33e64a789cb7c8c7d5779d95/users/e84ba8e4aac24fd78dac290b2c9f1c57/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:44.937907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:44.938486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:44.938772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:44.968981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] MFA Rules not processed for user `e84ba8e4aac24fd78dac290b2c9f1c57`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:44.970237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:45.034182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1072/2136] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:45.039242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.102678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f15207-b212-496a-8733-1044994570c1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.107459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.107993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.108269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.108588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.109192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.111077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.261423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1065/2137] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:45 2022] POST /identity/v3/projects => generated 383 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:45.265476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.277160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcc830-b85f-427a-b15a-4be9afb89950 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.349874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.350489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.350822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.351140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.351786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.353792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.459209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.460366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1073/2138] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:45 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:45.472222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b71b-6e0a-4e6a-bad1-e66b3ede692c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.478311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.478967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.479249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.479583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.480258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.481897 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.497418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1066/2139] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:45.498802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.510618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3712a12-2609-4fbd-8c66-98560a640ee3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.519700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155f02-c4ac-4afa-9e80-e40f80d8a8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.519700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155f02-c4ac-4afa-9e80-e40f80d8a8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.519700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155f02-c4ac-4afa-9e80-e40f80d8a8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.519700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155f02-c4ac-4afa-9e80-e40f80d8a8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.520406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2155f02-c4ac-4afa-9e80-e40f80d8a8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.521213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2155f02-c4ac-4afa-9e80-e40f80d8a8a0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.535882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1074/2140] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:45.538305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:45.540986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.550315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdc11-ae2f-40f3-a0ac-5665385bde81 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.556031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.556950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.557263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.557539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] PATH_INFO: `/v3/projects/aa1a2aa6285e4d16b76492f1d1090da9/users/92292fc443ff4cd69c3ea07490bfe28e/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.573327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1a2aa6285e4d16b76492f1d1090da9, user_id=92292fc443ff4cd69c3ea07490bfe28e, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.575991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.683649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1067/2141] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:45 2022] PUT /identity/v3/projects/aa1a2aa6285e4d16b76492f1d1090da9/users/92292fc443ff4cd69c3ea07490bfe28e/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:45.693797 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.694299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.695408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.735571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] MFA Rules not processed for user `92292fc443ff4cd69c3ea07490bfe28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:45.737048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:45.828814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1075/2142] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:45.833767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.888348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:45.925535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2707-137d-4476-bd32-860a9bf97f52 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.925535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.926136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.926424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.926707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.927367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.935556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.045391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1068/2143] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:45 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:46.049526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:46.064472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129851d7-1b30-4cb0-a790-82f46fa42b9d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:46.135120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:46.136069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.136346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.136674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.137691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:46.139954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.230949 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1076/2144] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:46 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:46.234958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:46.245934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e6257-3525-4057-9960-71eb27a9afbe admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:46.251815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:46.251815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.252038 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.252239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.253017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:46.254639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.294387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1069/2145] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:46.295745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:46.309186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61db9c-e0f8-4d0c-8a54-6606ef53da9e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:46.327584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:46.328289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.328573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.331832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] PATH_INFO: `/v3/projects/3e51167a955c413ea3cf4c113be6aefa/users/10a35827e5224d669eda5c52797f3330/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.351459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e51167a955c413ea3cf4c113be6aefa, user_id=10a35827e5224d669eda5c52797f3330, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:46.355601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.417550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1077/2146] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:46 2022] PUT /identity/v3/projects/3e51167a955c413ea3cf4c113be6aefa/users/10a35827e5224d669eda5c52797f3330/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:46.419423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:46.435027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd85b-5a4e-4f52-93bc-bd312d0d3361 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:46.511539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401837-c240-44d0-9674-9c665a5c5d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:46.512203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401837-c240-44d0-9674-9c665a5c5d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.512541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401837-c240-44d0-9674-9c665a5c5d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.512761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401837-c240-44d0-9674-9c665a5c5d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.513455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78401837-c240-44d0-9674-9c665a5c5d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:46.515204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78401837-c240-44d0-9674-9c665a5c5d74 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.526962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1070/2147] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:46.531234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:46.543715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6999d-2705-4a6a-b3ee-96c64123de89 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:46.549598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:46.550353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.550641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.551105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] PATH_INFO: `/v3/projects/3e51167a955c413ea3cf4c113be6aefa/users/10a35827e5224d669eda5c52797f3330/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.553101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e51167a955c413ea3cf4c113be6aefa, user_id=10a35827e5224d669eda5c52797f3330, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:46.556082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.567339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1078/2148] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:46 2022] PUT /identity/v3/projects/3e51167a955c413ea3cf4c113be6aefa/users/10a35827e5224d669eda5c52797f3330/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:46.579054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.579353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.579666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.610632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] MFA Rules not processed for user `10a35827e5224d669eda5c52797f3330`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:46.611882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:46.686787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1071/2149] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:46.699089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:46.700773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:46.723156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:46.782141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f960b5c-d96e-4064-9eab-1771cb2fe964 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:46.786786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702c4ab-7962-4162-833b-884bc4ee6fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:46.787615 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702c4ab-7962-4162-833b-884bc4ee6fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:46.787957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702c4ab-7962-4162-833b-884bc4ee6fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:46.790245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702c4ab-7962-4162-833b-884bc4ee6fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:46.864239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702c4ab-7962-4162-833b-884bc4ee6fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:46.866031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702c4ab-7962-4162-833b-884bc4ee6fe6 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:46.877289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1079/2150] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:49.060903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:49.062637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:49.077239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a4925-1762-45c9-9e62-3384a178f46d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:49.083035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:49.084075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:49.084075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:49.084303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:49.136148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:49.180775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:49.186181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:49.217234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1072/2151] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:50.729041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:50.729876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:50.730320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:50.759739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] MFA Rules not processed for user `1945fed03f73472db818c4e5ca015eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:50.761063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:50.834093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1080/2152] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:50.848509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:50.849963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:50.927006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e6e71-773d-4d50-95e8-9fe3c6f93e72 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:50.933585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:50.934244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:50.934524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:50.934824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:50.943370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:50.946057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:50.955655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1073/2153] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:51.137912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.139391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.239968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.249988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd774d-a435-4d59-882f-cc03ae923b83 service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.254680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.255460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.255752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.257901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.260272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a173732-72e6-45ad-ac9c-71b3538feeea None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.265744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.266385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.266852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.267277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] PATH_INFO: `/v3/users/cc943e3d68644e3a81dba6e765186258` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.267801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.268434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc943e3d68644e3a81dba6e765186258)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.269760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.270352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.272155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.282015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1074/2154] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:51.426482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.428110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.442748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cadd8-136b-4fdc-9f08-72ab702cad87 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.444354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc943e3d68644e3a81dba6e765186258'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.448957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.449713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.450095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.450542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] PATH_INFO: `/v3/users/9c6344c992cb4e408732b94b8eb5e4ad` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.452014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6344c992cb4e408732b94b8eb5e4ad)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.454043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.455685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc943e3d68644e3a81dba6e765186258'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.538947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc943e3d68644e3a81dba6e765186258'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.539835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc943e3d68644e3a81dba6e765186258'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.555053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-97e7103f-5d5a-4f0f-abef-3b6034547953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc943e3d68644e3a81dba6e765186258'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.562323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1081/2155] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:51 2022] DELETE /identity/v3/users/cc943e3d68644e3a81dba6e765186258 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:51.571275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.571567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.571866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.596350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:51.596675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.621903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6344c992cb4e408732b94b8eb5e4ad'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.680883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1082/2156] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:51.697806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.698165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6344c992cb4e408732b94b8eb5e4ad'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.700266 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.709056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6344c992cb4e408732b94b8eb5e4ad'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.710800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6344c992cb4e408732b94b8eb5e4ad'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.745772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1edbf906-d252-4598-9bc3-62d899841146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6344c992cb4e408732b94b8eb5e4ad'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:51.757917 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1075/2157] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:51 2022] DELETE /identity/v3/users/9c6344c992cb4e408732b94b8eb5e4ad => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:51.762723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.763481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.763776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.793443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:51.795081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.816622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.821559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d0ccf-4623-4120-acd8-54ce5487b7dd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.891360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.898633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1076/2158] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:51.906444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.908916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.916239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.917083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.917371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.917648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.919636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.929426 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683b9b-36d1-4f1b-a47a-8a60dbec4e67 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.935651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.937264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.937561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.937716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.937896 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.939608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.951876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.953649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.962628 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1083/2159] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:51.966961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1077/2160] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:52.581389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:52.582959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:52.603888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb048568-fb3f-4e1d-b28a-acda8337ced1 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:52.615915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:52.618178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:52.628940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:52.629751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:52.634658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:52.634843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] PATH_INFO: `/v3/projects/f67b31b5add540418173b931b3efdd49` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:52.637646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67b31b5add540418173b931b3efdd49)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:52.639791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:52.663852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98308846-dc43-4726-9be9-e547e4e78e1b service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:52.676122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:52.691652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:52.691652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:52.691652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] PATH_INFO: `/v3/projects/51ffd8208b694c3e81ce7ffa152596f1` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:52.691652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ffd8208b694c3e81ce7ffa152596f1)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:52.691652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:52.806422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67b31b5add540418173b931b3efdd49', 'request_id': 'req-c79f658e-7ef1-4df3-b04a-652ed49f31e2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.815194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ffd8208b694c3e81ce7ffa152596f1', 'request_id': 'req-3c72da84-e002-443a-b7d8-daea2fa1c175'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.823915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67b31b5add540418173b931b3efdd49', 'request_id': 'req-c79f658e-7ef1-4df3-b04a-652ed49f31e2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.827228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ffd8208b694c3e81ce7ffa152596f1', 'request_id': 'req-3c72da84-e002-443a-b7d8-daea2fa1c175'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.833573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67b31b5add540418173b931b3efdd49', 'request_id': 'req-c79f658e-7ef1-4df3-b04a-652ed49f31e2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.834258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ffd8208b694c3e81ce7ffa152596f1', 'request_id': 'req-3c72da84-e002-443a-b7d8-daea2fa1c175'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.907904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] The token cache is being invalidate because project 51ffd8208b694c3e81ce7ffa152596f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:52.908760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.910933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3c72da84-e002-443a-b7d8-daea2fa1c175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.913981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1078/2161] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:52 2022] DELETE /identity/v3/projects/51ffd8208b694c3e81ce7ffa152596f1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:52.919246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011750-9c03-49d9-9429-63c823a79210 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:52.919975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011750-9c03-49d9-9429-63c823a79210 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:52.920453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011750-9c03-49d9-9429-63c823a79210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:52.954815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0b011750-9c03-49d9-9429-63c823a79210 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:52.956075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0b011750-9c03-49d9-9429-63c823a79210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:52.972477 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] The token cache is being invalidate because project f67b31b5add540418173b931b3efdd49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:52.972774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.975846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c79f658e-7ef1-4df3-b04a-652ed49f31e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:52.977850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1084/2162] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:52 2022] DELETE /identity/v3/projects/f67b31b5add540418173b931b3efdd49 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:52.982673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fda395-ca76-4b04-90bf-dd72dd72f82e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:52.983228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fda395-ca76-4b04-90bf-dd72dd72f82e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:52.983503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fda395-ca76-4b04-90bf-dd72dd72f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.007413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f1fda395-ca76-4b04-90bf-dd72dd72f82e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:53.009087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f1fda395-ca76-4b04-90bf-dd72dd72f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:53.050046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1079/2163] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:53.058859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.059246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.059597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.082150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:53.089219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:53.089354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1085/2164] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:53.094839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.095123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.095394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.103293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1080/2165] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:53.108446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.121113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386ab89-7512-443f-ae91-a83449cc8931 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.126564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.127835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.128127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.129127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.129568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:53.129980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:53.134141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.134690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:53.134773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.144939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1086/2166] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:53.146174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.153027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1081/2167] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:53.162031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8f773-bab7-4a84-aa6b-d1e8fdce4456 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.163223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.169254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.170101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.170860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.182319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.182567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9efa-604c-48ab-9963-60c55a2b092d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.183114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:53.183432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.186962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.189719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.190958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.192045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.192283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.192928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.195799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.203306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:53.221183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1087/2168] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:53.222813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.234921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022430f8-bb76-4fd6-b61b-ca341372e92f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.240081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.240686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.240964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.241458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.242265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.245546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.312548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1082/2169] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:53.316666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.426266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327ac5-d4f8-4fbe-b39d-fe8d242494aa admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.433155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1088/2170] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:53.436631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.445205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.445205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.445205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.445205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.445650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.447526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.470310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:53.516802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d648549-1432-41d6-8fa5-567fef2bcc24 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.526118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.526850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.527281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.528095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.528753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.532196 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.587098 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1083/2171] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:53.591535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.603572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b47d39-7bf1-4cca-bd86-e01598a6c60c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.609067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.609696 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.611882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.611882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.611882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.612912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.627716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1084/2172] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:53.631494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.643509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1089/2173] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:53.646442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87101c0e-670d-4306-960a-bd085e1141e9 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.648706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.651642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.652330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.652529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.652821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.653655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.655757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.665941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4b7d0-1bcc-44b5-bfbe-95d60d65ea6a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.675416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1085/2174] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:53.677189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.678028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.678898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.679565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.680190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.681137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.683453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.693746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549eed7-162b-43b4-b0eb-8c3bb706167e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.700387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1090/2175] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:53.701393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.703979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05da99-a3c5-42f3-b211-38fabaa1865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.704666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05da99-a3c5-42f3-b211-38fabaa1865e admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.705118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05da99-a3c5-42f3-b211-38fabaa1865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.705399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05da99-a3c5-42f3-b211-38fabaa1865e admin admin] PATH_INFO: `/v3/projects/3e1a2560bd3d4e539a0be68d7d935850/users/8f2da42b6c6043bab05be1b1f9b7ea04/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.732646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05da99-a3c5-42f3-b211-38fabaa1865e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1a2560bd3d4e539a0be68d7d935850, user_id=8f2da42b6c6043bab05be1b1f9b7ea04, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.734209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e46e3-4d38-49a3-9bce-616352d1d588 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.735820 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05da99-a3c5-42f3-b211-38fabaa1865e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.745787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.757499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.757499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.757499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.757499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.760649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.773658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1091/2176] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:53.783248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.796412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e1fac-7f3f-4560-9fc9-18b4d7878f7d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:53.802557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6010b-cc6a-4af1-9103-eef136f5f97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:53.803377 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6010b-cc6a-4af1-9103-eef136f5f97f admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.804395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6010b-cc6a-4af1-9103-eef136f5f97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.804899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6010b-cc6a-4af1-9103-eef136f5f97f admin admin] PATH_INFO: `/v3/projects/3398675a6c1d44089f5fb2581ada3759/users/595d22773a4d46abb87f50f4c8c058a5/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.822043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1086/2177] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:53 2022] PUT /identity/v3/projects/3e1a2560bd3d4e539a0be68d7d935850/users/8f2da42b6c6043bab05be1b1f9b7ea04/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:53.823368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6010b-cc6a-4af1-9103-eef136f5f97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3398675a6c1d44089f5fb2581ada3759, user_id=595d22773a4d46abb87f50f4c8c058a5, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:53.826691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6010b-cc6a-4af1-9103-eef136f5f97f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:53.833763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.834188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.834539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.873104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] MFA Rules not processed for user `8f2da42b6c6043bab05be1b1f9b7ea04`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:53.874571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:53.911618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1092/2178] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:53 2022] PUT /identity/v3/projects/3398675a6c1d44089f5fb2581ada3759/users/595d22773a4d46abb87f50f4c8c058a5/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:53.916921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.917433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.917856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.951897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:53.964831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:53.972715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1087/2179] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:53.974420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.047792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1093/2180] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:54.057791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.059734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:54.064014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb572b-1448-49e0-9bdf-4a039acc2a0d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.071816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.072500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.072783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.073085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.073726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.076607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.144035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ab455-4e68-4c28-8988-359b1fee4d74 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.148146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.149049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.149163 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.149940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.160071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.161547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.176732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1094/2181] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:26:54.201704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1088/2182] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:54.203933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.222045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e851d78-55f7-44b7-9b6c-0b27860e17f7 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.309926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.317467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.317467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.317467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.317467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.317467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.454809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1095/2183] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:54 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:54.456930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.468428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b7de8-96c9-45cd-b643-c236635786ed admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.473377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.473999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.474355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.474562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.475213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.477541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.492199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1089/2184] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:54.496209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.510903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f4d57-3575-4fd1-a5cd-5a6f28c6d19f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.516312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fd419-76fb-41ff-a46e-729624c0ef0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.517346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fd419-76fb-41ff-a46e-729624c0ef0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.517626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fd419-76fb-41ff-a46e-729624c0ef0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.518355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fd419-76fb-41ff-a46e-729624c0ef0e admin admin] PATH_INFO: `/v3/projects/2d5e389251de4040a4869c510ee18c6f/users/cadeb07c3d3b4cb388257ced8fa31752/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.538831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fd419-76fb-41ff-a46e-729624c0ef0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5e389251de4040a4869c510ee18c6f, user_id=cadeb07c3d3b4cb388257ced8fa31752, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.542203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fd419-76fb-41ff-a46e-729624c0ef0e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.654261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1096/2185] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:54 2022] PUT /identity/v3/projects/2d5e389251de4040a4869c510ee18c6f/users/cadeb07c3d3b4cb388257ced8fa31752/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:54.654562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.685142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43394e0c-048c-4442-8d30-e062f65f44d5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.704871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e12bc-94ac-4087-9131-1e0ea0c5d30c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.705082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e12bc-94ac-4087-9131-1e0ea0c5d30c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.705256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e12bc-94ac-4087-9131-1e0ea0c5d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.745020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d98e12bc-94ac-4087-9131-1e0ea0c5d30c None None] MFA Rules not processed for user `1945fed03f73472db818c4e5ca015eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:54.746354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d98e12bc-94ac-4087-9131-1e0ea0c5d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:54.801115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.801743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.802234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.802515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.803340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.807134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.823064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1090/2186] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:54.827297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.830816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1097/2187] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:54.840378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf12a6-be1a-4d37-bf34-c9431edc0e69 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.845642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.846382 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.846731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.847079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] PATH_INFO: `/v3/projects/2d5e389251de4040a4869c510ee18c6f/users/cadeb07c3d3b4cb388257ced8fa31752/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.849386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5e389251de4040a4869c510ee18c6f, user_id=cadeb07c3d3b4cb388257ced8fa31752, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.853560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.872661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1091/2188] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:26:54 2022] PUT /identity/v3/projects/2d5e389251de4040a4869c510ee18c6f/users/cadeb07c3d3b4cb388257ced8fa31752/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:54.876530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b562e2-885f-4bf9-b743-98287103c7be None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.876819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b562e2-885f-4bf9-b743-98287103c7be None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.877120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b562e2-885f-4bf9-b743-98287103c7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.926612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-72b562e2-885f-4bf9-b743-98287103c7be None None] MFA Rules not processed for user `cadeb07c3d3b4cb388257ced8fa31752`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:54.928200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-72b562e2-885f-4bf9-b743-98287103c7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:55.063839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1098/2189] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:55.075402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:55.077667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:55.173514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258d8d-9e6f-4b55-a925-ef3dccbceabc admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:55.178116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:55.178757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:55.179175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:55.179647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:55.239156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:55.240612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:55.249821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1092/2190] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:55.350252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:55.883625 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:56.016720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:56.017334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:56.017798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:56.053702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] MFA Rules not processed for user `595d22773a4d46abb87f50f4c8c058a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:56.055552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:56.150276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1099/2191] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:56.159749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:56.161270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:56.179685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1409734-2083-4f41-b35a-e721664e43b7 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:56.184466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d170818-7d36-424b-bea5-5da2d6941c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:56.185362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d170818-7d36-424b-bea5-5da2d6941c87 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:56.185911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d170818-7d36-424b-bea5-5da2d6941c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:56.186200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d170818-7d36-424b-bea5-5da2d6941c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:56.198021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d170818-7d36-424b-bea5-5da2d6941c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:56.200010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d170818-7d36-424b-bea5-5da2d6941c87 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:56.224185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1093/2192] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:56.813155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:56.846429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:56.846429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:56.926078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307df2b8-2a46-4226-b5b0-36b921c08fed None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:56.932049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:56.932685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:56.932960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:56.933263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] PATH_INFO: `/v3/users/e84ba8e4aac24fd78dac290b2c9f1c57` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:56.934380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84ba8e4aac24fd78dac290b2c9f1c57)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:56.936324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:57.169596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84ba8e4aac24fd78dac290b2c9f1c57'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:57.176299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84ba8e4aac24fd78dac290b2c9f1c57'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:57.229627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84ba8e4aac24fd78dac290b2c9f1c57'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:57.230269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84ba8e4aac24fd78dac290b2c9f1c57'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:57.244405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84ba8e4aac24fd78dac290b2c9f1c57'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:57.251011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1100/2193] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:56 2022] DELETE /identity/v3/users/e84ba8e4aac24fd78dac290b2c9f1c57 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:57.255998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.256485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.257381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.278685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:57.279641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:57.281815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:57.346468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1094/2194] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:57.357235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:57.358883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:57.422754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05b66-dd04-4ea8-8fc9-d4e5731a0c26 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:57.426417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af9ed4-4c3b-4c81-8311-63cb7d1fc7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:57.427025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af9ed4-4c3b-4c81-8311-63cb7d1fc7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.427305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af9ed4-4c3b-4c81-8311-63cb7d1fc7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.427580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af9ed4-4c3b-4c81-8311-63cb7d1fc7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.436816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af9ed4-4c3b-4c81-8311-63cb7d1fc7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:57.438302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af9ed4-4c3b-4c81-8311-63cb7d1fc7c9 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:57.448595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1101/2195] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:57.969126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:57.969126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:57.980483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.980967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.981085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.990176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92523d7-f2e9-4535-b11d-417fbcd76835 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:57.995651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:57.996277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.996724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.997183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] PATH_INFO: `/v3/projects/40e87f1f33e64a789cb7c8c7d5779d95` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.998513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e87f1f33e64a789cb7c8c7d5779d95)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:58.001090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:58.022356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] MFA Rules not processed for user `595d22773a4d46abb87f50f4c8c058a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:58.024568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:58.094408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1102/2196] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:26:58.105261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e87f1f33e64a789cb7c8c7d5779d95', 'request_id': 'req-e9923221-1a8e-44ef-803b-8f4227b6c36e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.112053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e87f1f33e64a789cb7c8c7d5779d95', 'request_id': 'req-e9923221-1a8e-44ef-803b-8f4227b6c36e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.120661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e87f1f33e64a789cb7c8c7d5779d95', 'request_id': 'req-e9923221-1a8e-44ef-803b-8f4227b6c36e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.230172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] The token cache is being invalidate because project 40e87f1f33e64a789cb7c8c7d5779d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:58.230573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.231704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.232985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1095/2197] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:57 2022] DELETE /identity/v3/projects/40e87f1f33e64a789cb7c8c7d5779d95 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:58.236822 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:58.248979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733fa50-0374-4dc7-a1e5-dede6af1334d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:58.313052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:58.313719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:58.314002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:58.314279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] PATH_INFO: `/v3/users/92292fc443ff4cd69c3ea07490bfe28e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:58.317776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=92292fc443ff4cd69c3ea07490bfe28e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:58.320728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:58.341818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:58.343620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:58.357901 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9923221-1a8e-44ef-803b-8f4227b6c36e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:58.363129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:58.363964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:58.364404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:58.364831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] PATH_INFO: `/v3/users/8f2da42b6c6043bab05be1b1f9b7ea04` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:58.366880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2da42b6c6043bab05be1b1f9b7ea04)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:58.368883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:58.526130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92292fc443ff4cd69c3ea07490bfe28e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.541733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:58.542156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92292fc443ff4cd69c3ea07490bfe28e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.555011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2da42b6c6043bab05be1b1f9b7ea04'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.584002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:58.601288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92292fc443ff4cd69c3ea07490bfe28e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.602086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92292fc443ff4cd69c3ea07490bfe28e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.615894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0b81c8cf-1000-4d0c-a6d4-6ce784b1b6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92292fc443ff4cd69c3ea07490bfe28e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.621892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1103/2198] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:58 2022] DELETE /identity/v3/users/92292fc443ff4cd69c3ea07490bfe28e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:58.648181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2da42b6c6043bab05be1b1f9b7ea04'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.667388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2da42b6c6043bab05be1b1f9b7ea04'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.668453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2da42b6c6043bab05be1b1f9b7ea04'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.687714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2da42b6c6043bab05be1b1f9b7ea04'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:58.697070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1096/2199] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:58 2022] DELETE /identity/v3/users/8f2da42b6c6043bab05be1b1f9b7ea04 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:58.702146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:58.702146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:58.702146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:58.723619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:58.724868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:58.796418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1104/2200] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:58.810360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:58.812401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:58.879224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73a2ce-3482-42d4-93dd-181ed669b1cf admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:58.884273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:58.884938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:58.885288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:58.885560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:58.896430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:58.898535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:58.910653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1097/2201] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:59.395880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:59.397488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:59.415232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff10aa-801d-4fa3-be68-e21f46f4a5f2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:59.422549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:59.423282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:59.423563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:59.423878 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] PATH_INFO: `/v3/projects/aa1a2aa6285e4d16b76492f1d1090da9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:59.425098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1a2aa6285e4d16b76492f1d1090da9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:59.428291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:59.610553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1a2aa6285e4d16b76492f1d1090da9', 'request_id': 'req-c2c96891-1dc0-4c83-8458-1ff9485a616c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:59.621414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1a2aa6285e4d16b76492f1d1090da9', 'request_id': 'req-c2c96891-1dc0-4c83-8458-1ff9485a616c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:59.628424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1a2aa6285e4d16b76492f1d1090da9', 'request_id': 'req-c2c96891-1dc0-4c83-8458-1ff9485a616c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:59.751435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:59.753333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:59.753419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] The token cache is being invalidate because project aa1a2aa6285e4d16b76492f1d1090da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:26:59.753419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:59.754430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:26:59.756176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1105/2202] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:59 2022] DELETE /identity/v3/projects/aa1a2aa6285e4d16b76492f1d1090da9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:59.760958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:59.762401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:59.782562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e133a9-ce89-431f-a15d-dd02e36328cd service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:59.863056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c96891-1dc0-4c83-8458-1ff9485a616c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:59.870013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:59.870841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:59.871227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:59.871604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] PATH_INFO: `/v3/users/10a35827e5224d669eda5c52797f3330` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:59.873062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a35827e5224d669eda5c52797f3330)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:59.875283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:59.883931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:59.884640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:59.884968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:59.885455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] PATH_INFO: `/v3/projects/3e1a2560bd3d4e539a0be68d7d935850` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:59.886951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1a2560bd3d4e539a0be68d7d935850)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:59.889359 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:00.069224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1a2560bd3d4e539a0be68d7d935850', 'request_id': 'req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.082110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1a2560bd3d4e539a0be68d7d935850', 'request_id': 'req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.088484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a35827e5224d669eda5c52797f3330'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.093966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1a2560bd3d4e539a0be68d7d935850', 'request_id': 'req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.099897 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a35827e5224d669eda5c52797f3330'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.188048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] The token cache is being invalidate because project 3e1a2560bd3d4e539a0be68d7d935850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:27:00.188048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.188048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.188048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1098/2203] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:26:59 2022] DELETE /identity/v3/projects/3e1a2560bd3d4e539a0be68d7d935850 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:27:00.190603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:00.192132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:00.264846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a35827e5224d669eda5c52797f3330'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.265733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a35827e5224d669eda5c52797f3330'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.283302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a35827e5224d669eda5c52797f3330'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.303350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1106/2204] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:26:59 2022] DELETE /identity/v3/users/10a35827e5224d669eda5c52797f3330 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:27:00.304147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248a0b-7822-4f92-97f3-ddbc56f9d5b3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:00.378953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:00.382884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:00.382884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:00.382884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] PATH_INFO: `/v3/users/cadeb07c3d3b4cb388257ced8fa31752` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:00.382884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadeb07c3d3b4cb388257ced8fa31752)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:00.384378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:00.538946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadeb07c3d3b4cb388257ced8fa31752'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.620479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:00.652833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadeb07c3d3b4cb388257ced8fa31752'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.666408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadeb07c3d3b4cb388257ced8fa31752'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.667136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadeb07c3d3b4cb388257ced8fa31752'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.675977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:00.681063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadeb07c3d3b4cb388257ced8fa31752'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:00.687008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1099/2205] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:27:00 2022] DELETE /identity/v3/users/cadeb07c3d3b4cb388257ced8fa31752 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:27:01.017832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.019414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.034830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311650-61da-4d1d-bb81-f7161dadc2d4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.129347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.132390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.132390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.132390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] PATH_INFO: `/v3/projects/3e51167a955c413ea3cf4c113be6aefa` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.133939 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e51167a955c413ea3cf4c113be6aefa)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.135869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.224948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.227202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.242053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b713a0-0b10-45c3-816f-4001d5f7aa27 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.268232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e51167a955c413ea3cf4c113be6aefa', 'request_id': 'req-20ae5570-d30f-4c82-a826-146ff4c62ada'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.275450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e51167a955c413ea3cf4c113be6aefa', 'request_id': 'req-20ae5570-d30f-4c82-a826-146ff4c62ada'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.281876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e51167a955c413ea3cf4c113be6aefa', 'request_id': 'req-20ae5570-d30f-4c82-a826-146ff4c62ada'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.314250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.314887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.315308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.315577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] PATH_INFO: `/v3/projects/2d5e389251de4040a4869c510ee18c6f` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.316706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5e389251de4040a4869c510ee18c6f)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.318730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.349232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] The token cache is being invalidate because project 3e51167a955c413ea3cf4c113be6aefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:27:01.349818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.350560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-20ae5570-d30f-4c82-a826-146ff4c62ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.352290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1107/2206] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:27:01 2022] DELETE /identity/v3/projects/3e51167a955c413ea3cf4c113be6aefa => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:01.356793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78eb5-7b64-44ca-a911-63995d7edd0d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.357109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78eb5-7b64-44ca-a911-63995d7edd0d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.357388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78eb5-7b64-44ca-a911-63995d7edd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.377650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3de78eb5-7b64-44ca-a911-63995d7edd0d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:01.378871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3de78eb5-7b64-44ca-a911-63995d7edd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.400282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1108/2207] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:01.412670 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.413039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.413318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.436656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:01.437927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.452215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5e389251de4040a4869c510ee18c6f', 'request_id': 'req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.459097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5e389251de4040a4869c510ee18c6f', 'request_id': 'req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.468887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5e389251de4040a4869c510ee18c6f', 'request_id': 'req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.498939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:01.554492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1109/2208] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:01.558888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.568557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b286c47-94f8-4c91-99c0-9f610e05b13d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.576178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] The token cache is being invalidate because project 2d5e389251de4040a4869c510ee18c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:27:01.576178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.576178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b5ff29d6-2cec-4e98-9382-49a9c95be2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:01.576489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.576713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.577017 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.577308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.577981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1100/2209] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:27:01 2022] DELETE /identity/v3/projects/2d5e389251de4040a4869c510ee18c6f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:01.578436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:01.578826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.580446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.582769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0a0eb-2745-4577-8c0a-5f9589a1f172 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.583935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0a0eb-2745-4577-8c0a-5f9589a1f172 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.584400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0a0eb-2745-4577-8c0a-5f9589a1f172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.599078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1110/2210] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:01.601538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.603652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.618675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb5d1-68f5-4f41-bdb9-2879370657d4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.624022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.624864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.625240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-90d0a0eb-2745-4577-8c0a-5f9589a1f172 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:01.625371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.625748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.627963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.628094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-90d0a0eb-2745-4577-8c0a-5f9589a1f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.629076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.639660 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1101/2211] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:01.644479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:01.647432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.647784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.648064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.681320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:01.683033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.700583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1102/2212] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:01.705710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.717389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127806f-f8f6-4939-8932-540f19d22b69 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.726127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.727066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.727829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.728264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.729258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:01.729712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.731621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.734793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1111/2213] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:27:01.739111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.748109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1103/2214] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:27:01.752451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.770656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e084-797f-4e5c-81fe-a5d671f89352 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.826766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0737-c409-4d59-8ded-76c1d75693e1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.845431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.846149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.846434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.847507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.847507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.854161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.865200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.866792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.867350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.868043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.869179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.872613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.932716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1112/2215] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:27:01.936808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.947183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0908c-6ad1-4b2d-a312-71119178b3fb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.955306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.955306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.955543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1104/2216] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:27:01.955641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.955641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.955641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.955641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.958364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.969976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1113/2217] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:27:01.970964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.981587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bacd-9509-4113-888e-81c65faa16b1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.065165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71ccb8-00e1-42b2-b82f-b5571696d526 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.065847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.066660 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.066947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.067240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.068147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.071173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.077551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.077551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.077551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.077551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.078014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.078014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.083494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1114/2218] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:27:02.089826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.108431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcc280-8bb6-4955-aad9-ea75d1eb48a1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.113281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df635bcd-c500-485a-9bd8-d8410aa0c002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.113964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df635bcd-c500-485a-9bd8-d8410aa0c002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.114242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df635bcd-c500-485a-9bd8-d8410aa0c002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.114517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df635bcd-c500-485a-9bd8-d8410aa0c002 admin admin] PATH_INFO: `/v3/projects/4de66ca606494d0183ad2896ca9c1c2c/users/6d0cdab0c3d84b3480e103c004075d85/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.133693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df635bcd-c500-485a-9bd8-d8410aa0c002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de66ca606494d0183ad2896ca9c1c2c, user_id=6d0cdab0c3d84b3480e103c004075d85, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.135813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df635bcd-c500-485a-9bd8-d8410aa0c002 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.152983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1105/2219] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:27:02.156637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.168160 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156475fe-e17f-4aeb-b3f3-724100415919 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.172739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.173398 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.179488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.179488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.179488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.179488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.186027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1106/2220] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:27:02.189899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.196356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1115/2221] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:27:02 2022] PUT /identity/v3/projects/4de66ca606494d0183ad2896ca9c1c2c/users/6d0cdab0c3d84b3480e103c004075d85/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:02.201423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42564-95bb-4936-a8b3-0b1035232db8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.209161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.209543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.210684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.240782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] MFA Rules not processed for user `6d0cdab0c3d84b3480e103c004075d85`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:02.242675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:02.263413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.264017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.264303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.264575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.265246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.266946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.276858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1107/2222] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:27:02.280906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.290967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8f4f6-2acd-4677-b6eb-a357e5e28268 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.293677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:02.297872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58a1a-e5d5-476e-8b42-fa3d42486803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.298592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58a1a-e5d5-476e-8b42-fa3d42486803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.300069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58a1a-e5d5-476e-8b42-fa3d42486803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.300069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58a1a-e5d5-476e-8b42-fa3d42486803 admin admin] PATH_INFO: `/v3/projects/c90cc127739a456fa771a175c82ce728/users/e3173a40031c408195bc73af6d0651a7/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.308597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1116/2223] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:02.315097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe58a1a-e5d5-476e-8b42-fa3d42486803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90cc127739a456fa771a175c82ce728, user_id=e3173a40031c408195bc73af6d0651a7, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.316006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:02.317868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe58a1a-e5d5-476e-8b42-fa3d42486803 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.320952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.326114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:02.378167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1108/2224] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:27:02 2022] PUT /identity/v3/projects/c90cc127739a456fa771a175c82ce728/users/e3173a40031c408195bc73af6d0651a7/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:27:02.378460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fc0d-a311-47f0-8c8d-5f34163d6d77 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.390374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.390854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.391133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.426331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] MFA Rules not processed for user `e3173a40031c408195bc73af6d0651a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:02.427683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:02.442329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.442950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.443242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.443520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.501180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.501447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1109/2225] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:27:02.502668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.512631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1117/2226] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:27:02.518215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.520049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:02.538549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef7bb-c186-4d2f-b745-81e918ccbbd5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.540817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55918fff-5a51-4869-a265-4eb6a50ef16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.550837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55918fff-5a51-4869-a265-4eb6a50ef16b service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.553436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55918fff-5a51-4869-a265-4eb6a50ef16b service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.557043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55918fff-5a51-4869-a265-4eb6a50ef16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.569123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55918fff-5a51-4869-a265-4eb6a50ef16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.569123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55918fff-5a51-4869-a265-4eb6a50ef16b service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.596955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1110/2227] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:27:02.750227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:02.806978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:03.191187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:06.986434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:07.080251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09df6c-7e1f-47d9-b608-db453bb53030 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.086601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.087250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.087543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.087816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] PATH_INFO: `/v3/users/e3173a40031c408195bc73af6d0651a7` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.089002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3173a40031c408195bc73af6d0651a7)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.091000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.250801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3173a40031c408195bc73af6d0651a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:07.259357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3173a40031c408195bc73af6d0651a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:07.332485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3173a40031c408195bc73af6d0651a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:07.333131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3173a40031c408195bc73af6d0651a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:07.345653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3173a40031c408195bc73af6d0651a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:07.351553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1118/2228] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:27:06 2022] DELETE /identity/v3/users/e3173a40031c408195bc73af6d0651a7 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:07.356145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.356657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.356748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.376255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:07.377558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:07.439836 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1111/2229] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:07.456355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:07.457790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:07.509211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:07.523595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768691e-d983-4a90-ad69-7c8f54e58b3b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.527269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e640fd-7e7c-4cf1-82d6-c6673acee8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.530558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e640fd-7e7c-4cf1-82d6-c6673acee8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.530558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e640fd-7e7c-4cf1-82d6-c6673acee8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.530558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e640fd-7e7c-4cf1-82d6-c6673acee8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.538422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e640fd-7e7c-4cf1-82d6-c6673acee8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.539995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e640fd-7e7c-4cf1-82d6-c6673acee8e3 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.549787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1119/2230] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:27:08.196902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:08.204298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:08.230732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d7428-1007-481f-ac85-fe5ee6994287 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:08.230732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:08.231213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.231213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.231422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] PATH_INFO: `/v3/projects/c90cc127739a456fa771a175c82ce728` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.232575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90cc127739a456fa771a175c82ce728)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:08.234651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:08.327794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90cc127739a456fa771a175c82ce728', 'request_id': 'req-377d8582-2ac5-439c-a8b0-8b2644eeae31'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:08.333311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90cc127739a456fa771a175c82ce728', 'request_id': 'req-377d8582-2ac5-439c-a8b0-8b2644eeae31'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:08.340037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90cc127739a456fa771a175c82ce728', 'request_id': 'req-377d8582-2ac5-439c-a8b0-8b2644eeae31'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:08.394055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] The token cache is being invalidate because project c90cc127739a456fa771a175c82ce728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:27:08.394644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:08.394991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-377d8582-2ac5-439c-a8b0-8b2644eeae31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:27:08.396539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1112/2231] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:27:08 2022] DELETE /identity/v3/projects/c90cc127739a456fa771a175c82ce728 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:08.402068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.402464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.402622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.423132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:08.424332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:08.483991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1120/2232] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:08.492127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.492416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.492697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.513851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:08.515245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:08.526126 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1113/2233] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:08.536095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:08.546638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fc06e-4f06-4948-be83-5c4dfe5966b9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:08.551351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:08.551955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.552233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.552506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.553510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:08.554048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:08.555627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:08.573731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1121/2234] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:08.578114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:08.588502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ce58e-4f5c-4ab4-b761-9da622db9911 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:08.593192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:08.593781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.596029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.596029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.596029 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:08.596830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:08.728220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1114/2235] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:27:08 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:27:08.730730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:08.741621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da7269-2004-4372-ab7a-8f552fd744c0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:08.805887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:08.806509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.806790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.807066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.807705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:08.810040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:08.926836 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1122/2236] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:27:08 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:27:08.931584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:08.955230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef2e65-577c-445b-a6c6-9c87cdadbbe8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:08.973052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:08.973689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:08.973973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:08.974249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:08.975261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:08.977388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:09.002321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1115/2237] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:27:09.003445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:09.016915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c94f-bd3c-4a94-85a5-2831e42f2661 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:09.030150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5b05d-2762-48b4-8550-7160926d2217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:09.030659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5b05d-2762-48b4-8550-7160926d2217 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:09.030928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5b05d-2762-48b4-8550-7160926d2217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:09.031199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5b05d-2762-48b4-8550-7160926d2217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:09.032102 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5b05d-2762-48b4-8550-7160926d2217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:09.033806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5b05d-2762-48b4-8550-7160926d2217 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:09.049336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1123/2238] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:27:09.054158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:09.066711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2419-9ecb-4319-b1b1-6e0fb9ce34f7 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:09.078749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:09.080085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:09.080576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:09.081171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] PATH_INFO: `/v3/projects/fa43fdd55cde452bac8ce21371e1896c/users/f93554f193c44246bf7b859b9956b095/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:09.098559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:09.107742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa43fdd55cde452bac8ce21371e1896c, user_id=f93554f193c44246bf7b859b9956b095, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:09.111107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:09.204026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1116/2239] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:27:09 2022] PUT /identity/v3/projects/fa43fdd55cde452bac8ce21371e1896c/users/f93554f193c44246bf7b859b9956b095/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:09.217472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb907aba-6112-4ea6-ab1b-d8122e210791 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:09.217843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb907aba-6112-4ea6-ab1b-d8122e210791 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:09.218225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb907aba-6112-4ea6-ab1b-d8122e210791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:09.251654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cb907aba-6112-4ea6-ab1b-d8122e210791 None None] MFA Rules not processed for user `f93554f193c44246bf7b859b9956b095`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:09.252763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cb907aba-6112-4ea6-ab1b-d8122e210791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:09.319479 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:09.348690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1124/2240] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:09.360535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:09.362504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:09.450419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867065-9b7a-4071-9bbe-f058fd4fc364 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:09.454855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95e9f0-3d65-4d42-a071-ce99950d8d9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:09.455719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95e9f0-3d65-4d42-a071-ce99950d8d9e service glance] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:09.456154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95e9f0-3d65-4d42-a071-ce99950d8d9e service glance] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:09.456451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95e9f0-3d65-4d42-a071-ce99950d8d9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:09.467095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95e9f0-3d65-4d42-a071-ce99950d8d9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:09.468752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95e9f0-3d65-4d42-a071-ce99950d8d9e service glance] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:09.480977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1117/2241] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:27:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:27:14.131953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ffd92-663a-4a1b-954f-7ba0c9b0b67a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:14.132512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ffd92-663a-4a1b-954f-7ba0c9b0b67a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:14.132626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ffd92-663a-4a1b-954f-7ba0c9b0b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:14.152971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-515ffd92-663a-4a1b-954f-7ba0c9b0b67a None None] MFA Rules not processed for user `4dfd3ee79da943b189824630b5c393ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:14.154211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-515ffd92-663a-4a1b-954f-7ba0c9b0b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:14.211131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1125/2242] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:19.676881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:21.596252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:21.596821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:21.597103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:21.634047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] MFA Rules not processed for user `c9577bc4c4c546ca9fc220ebd3a16346`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:21.635526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:21.732549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1118/2243] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:24.204388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=35 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:31.723697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=25 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:32.520929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610c83-3656-40a6-8cd0-0563b76eb8c5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:32.520929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610c83-3656-40a6-8cd0-0563b76eb8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:32.520929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610c83-3656-40a6-8cd0-0563b76eb8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:32.542248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-41610c83-3656-40a6-8cd0-0563b76eb8c5 None None] MFA Rules not processed for user `595d22773a4d46abb87f50f4c8c058a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:32.543508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-41610c83-3656-40a6-8cd0-0563b76eb8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:32.616983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1126/2244] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:27:42.607335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=25 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:03.263894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:03.265906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:03.332584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bb298-c270-4faa-bc17-96fe1d44ce0c None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:03.337395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:03.338030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:03.338313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:03.338588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] PATH_INFO: `/v3/users/6d0cdab0c3d84b3480e103c004075d85` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:03.339715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0cdab0c3d84b3480e103c004075d85)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:03.341795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:03.563523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0cdab0c3d84b3480e103c004075d85'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:03.663673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0cdab0c3d84b3480e103c004075d85'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:03.673470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0cdab0c3d84b3480e103c004075d85'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:03.673470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0cdab0c3d84b3480e103c004075d85'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:03.688145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0cdab0c3d84b3480e103c004075d85'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:03.694685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1119/2245] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:28:03 2022] DELETE /identity/v3/users/6d0cdab0c3d84b3480e103c004075d85 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:03.704463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:03.705090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:03.705090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:03.729914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:03.734488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:03.796257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1127/2246] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:03.806420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:03.809142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:03.866086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e2e2b-4c2e-4943-9101-621e1c80a939 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:03.869877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:03.870484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:03.870766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:03.871041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:03.881891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:03.883326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:03.894573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1120/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:28:04.294517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:04.296476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:04.309892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114fdbf-44f0-4474-87f4-9f3d0f9a5891 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:04.318686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:04.319564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:04.320025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:04.320282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] PATH_INFO: `/v3/projects/4de66ca606494d0183ad2896ca9c1c2c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:04.322345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de66ca606494d0183ad2896ca9c1c2c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:04.324616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:04.475520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de66ca606494d0183ad2896ca9c1c2c', 'request_id': 'req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:04.481449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de66ca606494d0183ad2896ca9c1c2c', 'request_id': 'req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:04.488277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de66ca606494d0183ad2896ca9c1c2c', 'request_id': 'req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:04.572129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] The token cache is being invalidate because project 4de66ca606494d0183ad2896ca9c1c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:28:04.572357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:04.572981 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cb6c0fb6-af27-433a-9a7b-a2d13515a2e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:04.574517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1128/2248] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:28:04 2022] DELETE /identity/v3/projects/4de66ca606494d0183ad2896ca9c1c2c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:05.513338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:05.515249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:05.530210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498a2dc-0ad2-4306-84f2-affe50bc3fe4 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:05.554602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:05.598905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:05.599541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:05.599826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:05.600101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] PATH_INFO: `/v3/users/f93554f193c44246bf7b859b9956b095` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:05.601261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93554f193c44246bf7b859b9956b095)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:05.603243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:05.792005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93554f193c44246bf7b859b9956b095'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:05.863388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93554f193c44246bf7b859b9956b095'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:05.871027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93554f193c44246bf7b859b9956b095'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:05.873918 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93554f193c44246bf7b859b9956b095'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:05.885655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93554f193c44246bf7b859b9956b095'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:05.893090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1121/2249] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:28:05 2022] DELETE /identity/v3/users/f93554f193c44246bf7b859b9956b095 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:05.899782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:05.900257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:05.900366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:05.922990 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:05.924317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:05.988725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1129/2250] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:28:05.998586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:06.000144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:06.059459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc6c3-bc94-4f24-8abd-2ac717e362ac admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:06.064790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2ef70-c732-474e-8a2e-cc75494df899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:06.065471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2ef70-c732-474e-8a2e-cc75494df899 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:06.065753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2ef70-c732-474e-8a2e-cc75494df899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:06.066025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2ef70-c732-474e-8a2e-cc75494df899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:06.077565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2ef70-c732-474e-8a2e-cc75494df899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:06.079303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2ef70-c732-474e-8a2e-cc75494df899 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:06.092584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1122/2251] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:28:06.602023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:06.603838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:06.618647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92d92-a9fb-45e3-a96f-8b42c5bd8058 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:06.628035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:06.628035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:06.628035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:06.628035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] PATH_INFO: `/v3/projects/fa43fdd55cde452bac8ce21371e1896c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:06.628737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa43fdd55cde452bac8ce21371e1896c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:06.631033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:06.659097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:06.726882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa43fdd55cde452bac8ce21371e1896c', 'request_id': 'req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:06.732815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa43fdd55cde452bac8ce21371e1896c', 'request_id': 'req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:06.739023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa43fdd55cde452bac8ce21371e1896c', 'request_id': 'req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:06.800550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] The token cache is being invalidate because project fa43fdd55cde452bac8ce21371e1896c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:28:06.800840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:06.801518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:06.803063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1130/2252] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:28:06 2022] DELETE /identity/v3/projects/fa43fdd55cde452bac8ce21371e1896c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:16.087314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:16.880888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:21.355871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720f5d-8392-4363-9e56-2b120d3fc3b9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.356319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720f5d-8392-4363-9e56-2b120d3fc3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.356406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720f5d-8392-4363-9e56-2b120d3fc3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.376808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4e720f5d-8392-4363-9e56-2b120d3fc3b9 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:21.378035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4e720f5d-8392-4363-9e56-2b120d3fc3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:21.436282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1123/2253] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:21.485269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:21.486741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:21.556276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1ad4-a1d9-471b-ad0a-9492ca72b421 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:21.561679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:21.562289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.562568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.562840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.570975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:21.574813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:21.580624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1131/2254] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:28:25.482087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62e4fc-ed38-4125-bf0a-9b4a2325cd66 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:25.482737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62e4fc-ed38-4125-bf0a-9b4a2325cd66 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:25.483419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62e4fc-ed38-4125-bf0a-9b4a2325cd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:25.503166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5b62e4fc-ed38-4125-bf0a-9b4a2325cd66 None None] MFA Rules not processed for user `d4e91c327d0c4b5dbae54cb915aca3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:25.504652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5b62e4fc-ed38-4125-bf0a-9b4a2325cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:25.568540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1124/2255] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:26.928453 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:26.930020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:27.024131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7358a12-41f9-4fa7-bf68-81684220add2 service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:27.033968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:27.035041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:27.035423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:27.035586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:27.037487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:27.050552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:27.052097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:27.063855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1132/2256] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:28:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:28:35.561177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=50 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:37.058810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=64 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:37.744905 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:37.745438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:37.745544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:37.771343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:37.771343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:37.778109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1125/2257] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 10 13:28:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:37.797252 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:37.808558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045397-7337-4b83-b6a3-6668d1c3d1c3 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:37.813833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:37.814459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:37.814732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:37.814997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:37.823793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:37.829638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:37.834826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1133/2258] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:28:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:28:40.852732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:40.852732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:40.921509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fd930-de1b-49a9-abff-ab79e9fa9cd9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:40.926031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:40.926657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:40.926942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:40.927217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] PATH_INFO: `/v3/users/595d22773a4d46abb87f50f4c8c058a5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:40.928447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=595d22773a4d46abb87f50f4c8c058a5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:40.930386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:41.122186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '595d22773a4d46abb87f50f4c8c058a5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.199925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '595d22773a4d46abb87f50f4c8c058a5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.208279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '595d22773a4d46abb87f50f4c8c058a5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.208965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '595d22773a4d46abb87f50f4c8c058a5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.221325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-18471ad1-6556-4c1f-bffc-4a072b759414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '595d22773a4d46abb87f50f4c8c058a5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.227250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1126/2259] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:28:40 2022] DELETE /identity/v3/users/595d22773a4d46abb87f50f4c8c058a5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:28:41.437199 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:41.447810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10298df-514c-4398-82f2-8cc2d6785b79 service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:41.505332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:41.506040 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:41.506320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:41.506619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] PATH_INFO: `/v3/projects/3398675a6c1d44089f5fb2581ada3759` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:41.507728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3398675a6c1d44089f5fb2581ada3759)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:41.509706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:41.606283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3398675a6c1d44089f5fb2581ada3759', 'request_id': 'req-aaac6865-a278-4f21-94bb-5852cc59c530'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.612534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3398675a6c1d44089f5fb2581ada3759', 'request_id': 'req-aaac6865-a278-4f21-94bb-5852cc59c530'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.618765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3398675a6c1d44089f5fb2581ada3759', 'request_id': 'req-aaac6865-a278-4f21-94bb-5852cc59c530'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.702825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] The token cache is being invalidate because project 3398675a6c1d44089f5fb2581ada3759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:28:41.703100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.703940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aaac6865-a278-4f21-94bb-5852cc59c530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:28:41.705365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1134/2260] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:28:41 2022] DELETE /identity/v3/projects/3398675a6c1d44089f5fb2581ada3759 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:28:41.717681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:41.717681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:41.717681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:41.733884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:41.735080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:41.792877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1127/2261] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:28:41.801078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:41.801367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:41.801639 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:41.821157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:41.822345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:41.830799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1135/2262] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:41.834773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:41.844614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f25c8-587d-4e3b-968c-78c72353fecc None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:41.849364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:41.849957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:41.850239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:41.850511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:41.851189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:41.851494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:41.853201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:41.861826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:41.865077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1128/2263] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:41.869333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:41.879323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f4a11-935f-4cb6-a54c-1fdbfde2a127 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:41.884043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:41.884635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:41.884914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:41.885219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:41.885852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:41.887744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:41.995560 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1136/2264] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:28:41 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:28:41.999805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:42.009671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc3d94-0074-42e6-bf91-254be95440bb admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:42.066314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:42.066913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.067193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.067504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.068115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:42.070159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:42.160950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1129/2265] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:28:41 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:28:42.165223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:42.181830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f0b5-2d08-4da9-9415-a7d2d995ade0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:42.186520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:42.187112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.187392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.187666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.188306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:42.189955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:42.200085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1137/2266] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:28:42.205403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:42.215531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca723-df34-4d47-89c1-760859bc91ca admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:42.220942 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384b788-94cf-4d4f-afb9-75bf423d4157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:42.221557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384b788-94cf-4d4f-afb9-75bf423d4157 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.221837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384b788-94cf-4d4f-afb9-75bf423d4157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.222106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384b788-94cf-4d4f-afb9-75bf423d4157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.222741 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384b788-94cf-4d4f-afb9-75bf423d4157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:42.224323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384b788-94cf-4d4f-afb9-75bf423d4157 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:42.234833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1130/2267] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:28:42.238870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:42.248875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071aea0-380f-4edc-a869-c7807ebbd3e9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:42.253609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:42.254274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.254552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.254826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] PATH_INFO: `/v3/projects/6ccc46e1ccb04fb386367a71ad13f008/users/379f81ea933d4207865a8d1f299b5634/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.269421 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccc46e1ccb04fb386367a71ad13f008, user_id=379f81ea933d4207865a8d1f299b5634, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:42.272068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:42.319407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1138/2268] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:28:42 2022] PUT /identity/v3/projects/6ccc46e1ccb04fb386367a71ad13f008/users/379f81ea933d4207865a8d1f299b5634/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:42.324319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab018-4802-4be0-9086-5e28491d340f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.324612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab018-4802-4be0-9086-5e28491d340f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.324884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab018-4802-4be0-9086-5e28491d340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.344652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b8bab018-4802-4be0-9086-5e28491d340f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:42.345902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b8bab018-4802-4be0-9086-5e28491d340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:42.405565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1131/2269] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:42.415562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:42.417291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:42.432409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:42.475532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cec070-6d7e-49c9-a6e6-f632f4947781 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:42.479217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:42.479847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.480127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.480397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.490664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:42.492081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:42.504929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1139/2270] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:28:47.516447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:47.522414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:47.522808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:47.565677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] MFA Rules not processed for user `379f81ea933d4207865a8d1f299b5634`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:47.567465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:47.671057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1132/2271] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:47.676032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:47.688549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ad838-23f1-415a-86ab-9c28a8e4a7e4 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:47.694302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:47.695132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:47.695514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:47.695903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:47.696661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:47.699066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:47.820832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1140/2272] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:28:47 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:28:47.826980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:47.840469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268f68c-9064-4b85-95d4-f2f12481acae None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:47.849370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:47.930853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:47.931633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:47.932022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:47.932403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:47.933208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:47.935783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:48.040956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1133/2273] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:28:47 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:28:48.046189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:48.062837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e582bb4-1309-4c5b-88fc-22e2c7eb0fae admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:48.068543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:48.069312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:48.069665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:48.070007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:48.070844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:48.072570 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:48.084172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1141/2274] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:28:48.088789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:48.100139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820b66-7e04-402a-b85d-1e73091d1dd2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:48.105072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:48.105745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:48.106388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:48.107250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] PATH_INFO: `/v3/projects/9baf1533c0ec48b3b1ac29b251a9e5b9/users/3dc49964b34444a492ea183dc933b9e1/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:48.129989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baf1533c0ec48b3b1ac29b251a9e5b9, user_id=3dc49964b34444a492ea183dc933b9e1, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:48.134459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:48.213862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1134/2275] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:28:48 2022] PUT /identity/v3/projects/9baf1533c0ec48b3b1ac29b251a9e5b9/users/3dc49964b34444a492ea183dc933b9e1/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:28:48.219035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:48.229767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2049f0-32ed-4267-b51d-ffdf0aaa8333 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:48.300103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51c16b-f2f8-44e1-84d2-fcf6dfd814f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:48.300768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51c16b-f2f8-44e1-84d2-fcf6dfd814f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:48.301114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51c16b-f2f8-44e1-84d2-fcf6dfd814f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:48.301391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51c16b-f2f8-44e1-84d2-fcf6dfd814f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:48.302034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51c16b-f2f8-44e1-84d2-fcf6dfd814f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:48.303824 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51c16b-f2f8-44e1-84d2-fcf6dfd814f9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:48.317389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1142/2276] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:28:48.319352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:48.330400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50ab37-4e2e-4bc8-bfcb-ce065f74c0b2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:48.335796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:48.336474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:48.336756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:48.337149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] PATH_INFO: `/v3/projects/9baf1533c0ec48b3b1ac29b251a9e5b9/users/3dc49964b34444a492ea183dc933b9e1/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:48.339208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baf1533c0ec48b3b1ac29b251a9e5b9, user_id=3dc49964b34444a492ea183dc933b9e1, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:48.342472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:48.360162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1135/2277] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:28:48 2022] PUT /identity/v3/projects/9baf1533c0ec48b3b1ac29b251a9e5b9/users/3dc49964b34444a492ea183dc933b9e1/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:55.248323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab084f2-2d23-4440-ab65-bf37a758f948 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:55.248872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab084f2-2d23-4440-ab65-bf37a758f948 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:55.248872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab084f2-2d23-4440-ab65-bf37a758f948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:55.289388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dab084f2-2d23-4440-ab65-bf37a758f948 None None] MFA Rules not processed for user `3dc49964b34444a492ea183dc933b9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:55.291279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dab084f2-2d23-4440-ab65-bf37a758f948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:55.394267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1143/2278] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:55.404215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:55.409353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:55.483588 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0bb3c-97e7-427b-99f9-fe0dc57e9bd6 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:55.488147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:55.488762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:55.489100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:55.489374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:55.580574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:55.582164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:55.595932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1136/2279] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:28:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:28:55.902425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08a549-b7e9-4a2f-8024-9edbaeec743e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:55.903005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08a549-b7e9-4a2f-8024-9edbaeec743e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:55.903987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08a549-b7e9-4a2f-8024-9edbaeec743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:55.930055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:55.954597 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ec08a549-b7e9-4a2f-8024-9edbaeec743e None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:55.962295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ec08a549-b7e9-4a2f-8024-9edbaeec743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:56.068258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1144/2280] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:56.087107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:56.088054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:56.109387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:56.172713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaef98c-a03c-4516-af65-83dee67e7122 service glance] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:56.184433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f933d-78e8-444a-b31d-f289bc715f3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:56.186022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f933d-78e8-444a-b31d-f289bc715f3d service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:56.186438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f933d-78e8-444a-b31d-f289bc715f3d service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:56.186724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f933d-78e8-444a-b31d-f289bc715f3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:56.198513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f933d-78e8-444a-b31d-f289bc715f3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:56.201416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f933d-78e8-444a-b31d-f289bc715f3d service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:56.212546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1137/2281] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:28:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:29:06.254432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=23 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:06.367170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=25 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:07.737478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:07.737944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:07.738026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:07.757705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] MFA Rules not processed for user `379f81ea933d4207865a8d1f299b5634`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:07.762169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:07.766879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1145/2282] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:11.911962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899c14c-888c-4b2b-bf82-18b7a618278a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:11.912432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899c14c-888c-4b2b-bf82-18b7a618278a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:11.912526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899c14c-888c-4b2b-bf82-18b7a618278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:11.951490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5899c14c-888c-4b2b-bf82-18b7a618278a None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:11.952662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5899c14c-888c-4b2b-bf82-18b7a618278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:11.962317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1138/2283] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:11.979847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:12.012567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa57c80-45e5-40c9-b0ff-e87423fccaa9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:12.022209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:12.023272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:12.023707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:12.024198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:12.040183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:12.045937 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:12.057747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1146/2284] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:29:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:29:13.397392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994253f-6a74-47e6-8a72-65149d0af406 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:13.398232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994253f-6a74-47e6-8a72-65149d0af406 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:13.398659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994253f-6a74-47e6-8a72-65149d0af406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:13.446761 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1994253f-6a74-47e6-8a72-65149d0af406 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:13.454633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1994253f-6a74-47e6-8a72-65149d0af406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:13.483361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1139/2285] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:29:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:13.526193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:13.564350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85d4eb-2eb2-4d93-9a41-578408b5a7f8 service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:13.598050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26ac5-0998-45b3-8c57-eb7daf48b3d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:13.599652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26ac5-0998-45b3-8c57-eb7daf48b3d3 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:13.600136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26ac5-0998-45b3-8c57-eb7daf48b3d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:13.600693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26ac5-0998-45b3-8c57-eb7daf48b3d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:13.627731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26ac5-0998-45b3-8c57-eb7daf48b3d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:13.630749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26ac5-0998-45b3-8c57-eb7daf48b3d3 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:13.644501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1147/2286] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:29:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:29:20.786396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22ad38-28e2-4d88-83b9-3b41a5d68293 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:20.787418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22ad38-28e2-4d88-83b9-3b41a5d68293 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:20.787916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22ad38-28e2-4d88-83b9-3b41a5d68293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:20.834071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6c22ad38-28e2-4d88-83b9-3b41a5d68293 None None] MFA Rules not processed for user `c9577bc4c4c546ca9fc220ebd3a16346`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:20.836408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6c22ad38-28e2-4d88-83b9-3b41a5d68293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:21.007112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1140/2287] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:23.639545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=15 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:30.982668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=31 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:35.547967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3267b53-0bf5-4ac0-b86f-70c346c2ab41 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:35.548579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3267b53-0bf5-4ac0-b86f-70c346c2ab41 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:35.548877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3267b53-0bf5-4ac0-b86f-70c346c2ab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:35.576098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a3267b53-0bf5-4ac0-b86f-70c346c2ab41 None None] MFA Rules not processed for user `4dfd3ee79da943b189824630b5c393ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:35.578056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a3267b53-0bf5-4ac0-b86f-70c346c2ab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:35.668919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1148/2288] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:39.910161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8ca44-153b-4549-a106-1eb0417df533 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:39.910651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8ca44-153b-4549-a106-1eb0417df533 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:39.910744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8ca44-153b-4549-a106-1eb0417df533 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:39.913070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1141/2289] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:39.920010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69c37d-6f1b-4f1a-b022-e12f5500a68e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:39.920759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69c37d-6f1b-4f1a-b022-e12f5500a68e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:39.920759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69c37d-6f1b-4f1a-b022-e12f5500a68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:39.943439 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2d69c37d-6f1b-4f1a-b022-e12f5500a68e None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:39.944856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2d69c37d-6f1b-4f1a-b022-e12f5500a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:40.036910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1149/2290] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:40.052567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8ca44-153b-4549-a106-1eb0417df533 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:40.054732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c4a8ca44-153b-4549-a106-1eb0417df533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:40.070585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8ca44-153b-4549-a106-1eb0417df533 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:40.075136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7cfea-80f1-4052-8aac-da00d603ce40 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:40.075795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7cfea-80f1-4052-8aac-da00d603ce40 service glance] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.076083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7cfea-80f1-4052-8aac-da00d603ce40 service glance] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.076362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7cfea-80f1-4052-8aac-da00d603ce40 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.087675 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7cfea-80f1-4052-8aac-da00d603ce40 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:40.089971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7cfea-80f1-4052-8aac-da00d603ce40 service glance] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:40.101131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1142/2291] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:40.327372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2f67-fa60-4aae-8da2-f613a5b39fab None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.327662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2f67-fa60-4aae-8da2-f613a5b39fab None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.328082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2f67-fa60-4aae-8da2-f613a5b39fab None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.329470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1150/2292] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:40.334541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e940aa-804e-4451-a57b-820f4996cb03 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.334921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e940aa-804e-4451-a57b-820f4996cb03 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.335198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e940aa-804e-4451-a57b-820f4996cb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.356549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d9e940aa-804e-4451-a57b-820f4996cb03 None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:40.356549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d9e940aa-804e-4451-a57b-820f4996cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:40.418293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1143/2293] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:29:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:40.430951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa2f67-fa60-4aae-8da2-f613a5b39fab None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:40.445850 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa2f67-fa60-4aae-8da2-f613a5b39fab None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:40.450972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7910ce-41c3-460b-b33b-f4206240aea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:40.452240 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7910ce-41c3-460b-b33b-f4206240aea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.452764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7910ce-41c3-460b-b33b-f4206240aea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.453261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7910ce-41c3-460b-b33b-f4206240aea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.462158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7910ce-41c3-460b-b33b-f4206240aea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:40.463869 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7910ce-41c3-460b-b33b-f4206240aea9 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:40.485794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1151/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:40.599302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2bd12-bca8-47d9-ada0-9bf4e4dbb2f8 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.599855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2bd12-bca8-47d9-ada0-9bf4e4dbb2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.600133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2bd12-bca8-47d9-ada0-9bf4e4dbb2f8 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.601644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1144/2295] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:40.608975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.609205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.609529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.629784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:40.630980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:40.642880 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1152/2296] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:29:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:29:40.651017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2bd12-bca8-47d9-ada0-9bf4e4dbb2f8 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:40.663797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2bd12-bca8-47d9-ada0-9bf4e4dbb2f8 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:40.668144 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:40.668913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:40.669227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:40.669501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:40.679959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:40.681434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:40.693225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1145/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:42.218851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:42.221644 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:42.309522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78acfc-8890-4a97-aa34-881e0fdea913 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:42.314653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:42.316605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:42.316605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:42.316605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] PATH_INFO: `/v3/users/d4e91c327d0c4b5dbae54cb915aca3e6` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:42.317500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e91c327d0c4b5dbae54cb915aca3e6)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:42.319456 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:42.343272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:42.514876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e91c327d0c4b5dbae54cb915aca3e6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:42.522171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e91c327d0c4b5dbae54cb915aca3e6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:42.599678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e91c327d0c4b5dbae54cb915aca3e6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:42.600748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e91c327d0c4b5dbae54cb915aca3e6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:42.618618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-386512ed-403a-4f65-86c0-e81186485510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e91c327d0c4b5dbae54cb915aca3e6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:42.643432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1153/2298] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:29:42 2022] DELETE /identity/v3/users/d4e91c327d0c4b5dbae54cb915aca3e6 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:29:42.921265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:42.938985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084d874-cb96-4807-8bea-00e2cb49e41d service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.033652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.036654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.036654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.037323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] PATH_INFO: `/v3/projects/e43f6ad26c6a4dcdb27c9d9ba706cd0e` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.038709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43f6ad26c6a4dcdb27c9d9ba706cd0e)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:43.041141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:43.205222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43f6ad26c6a4dcdb27c9d9ba706cd0e', 'request_id': 'req-788ac5ad-6eef-4607-a0ac-a7ced08d7545'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:43.211662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43f6ad26c6a4dcdb27c9d9ba706cd0e', 'request_id': 'req-788ac5ad-6eef-4607-a0ac-a7ced08d7545'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:43.219488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43f6ad26c6a4dcdb27c9d9ba706cd0e', 'request_id': 'req-788ac5ad-6eef-4607-a0ac-a7ced08d7545'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:43.301637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] The token cache is being invalidate because project e43f6ad26c6a4dcdb27c9d9ba706cd0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:29:43.301980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:43.302738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-788ac5ad-6eef-4607-a0ac-a7ced08d7545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:43.304754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1146/2299] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:29:42 2022] DELETE /identity/v3/projects/e43f6ad26c6a4dcdb27c9d9ba706cd0e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:29:43.309346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.309780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.309871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.329921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:43.331136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:43.392916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1154/2300] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:43.402425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.403113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.403713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.440467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:43.441737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:43.456602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1147/2301] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:43.460636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:43.471059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b9097-9547-4843-8ad3-8d53f97fc565 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.476305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.477259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.487800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.487800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.487800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:29:43.487800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:43.487800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:43.501284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1155/2302] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:29:43.510852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:43.524699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe786f-e0ed-4c5b-9ef1-ca933547d730 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.530315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.532003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.532436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.533273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.534069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:43.536868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:43.544839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:43.671484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1148/2303] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:29:43 2022] POST /identity/v3/projects => generated 399 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:29:43.672766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:43.684044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdf3d0-4586-4b8c-8f24-24dcf1b5d329 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.764414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.765313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.765669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.765950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.766586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:43.769759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:43.856068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1156/2304] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:29:43 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:29:43.857739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:43.869363 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7749346-951f-4294-a1ec-ce884e664df2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.875430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.876053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.876335 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.876663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.878081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:43.879933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:43.895200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1149/2305] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:29:43.896406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:43.907974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144b230-bcc3-4044-9a92-f92f4085be72 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.913929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c57d8a-3789-4e16-8b39-bdae4c73bac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.922904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c57d8a-3789-4e16-8b39-bdae4c73bac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.923227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c57d8a-3789-4e16-8b39-bdae4c73bac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.923510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c57d8a-3789-4e16-8b39-bdae4c73bac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.924794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c57d8a-3789-4e16-8b39-bdae4c73bac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:43.928036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c57d8a-3789-4e16-8b39-bdae4c73bac1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:43.942536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1157/2306] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:29:43.946706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:43.960559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df470a8c-5987-44a4-9ca5-a58ce342b469 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:43.966299 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:43.967251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:43.967656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:43.968019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] PATH_INFO: `/v3/projects/67cecf7022af4316922c5173db3a3636/users/4a4c7d0ba47c452a8ab431c41c00ac04/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:43.998984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] RBAC: Authorizing `identity:create_grant(project_id=67cecf7022af4316922c5173db3a3636, user_id=4a4c7d0ba47c452a8ab431c41c00ac04, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:44.002263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:44.114047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1150/2307] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:29:43 2022] PUT /identity/v3/projects/67cecf7022af4316922c5173db3a3636/users/4a4c7d0ba47c452a8ab431c41c00ac04/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:29:44.125191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dd0bd-7379-4f2c-a612-debce70db538 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:44.125713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dd0bd-7379-4f2c-a612-debce70db538 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:44.126125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dd0bd-7379-4f2c-a612-debce70db538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:44.139349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:44.185607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ec5dd0bd-7379-4f2c-a612-debce70db538 None None] MFA Rules not processed for user `4a4c7d0ba47c452a8ab431c41c00ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:44.191295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ec5dd0bd-7379-4f2c-a612-debce70db538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:44.290378 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1158/2308] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:44.302042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:44.304175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:44.379255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cde60-a4c5-4cd2-b0ed-b7245c90288d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:44.384304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:44.385120 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:44.385347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:44.392393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:44.395953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:44.397469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:44.407549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1151/2309] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:46.174606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:46.175061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:46.175292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:46.195727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:46.198258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:46.258852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1159/2310] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:29:46.271347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:46.272749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:46.347542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41d10-547c-4e1a-b6c4-aa0a66eceb86 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:46.353258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480822c5-fca6-4417-8346-641a99149912 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:46.353895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480822c5-fca6-4417-8346-641a99149912 service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:46.354179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480822c5-fca6-4417-8346-641a99149912 service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:46.355001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480822c5-fca6-4417-8346-641a99149912 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:46.364275 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480822c5-fca6-4417-8346-641a99149912 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:46.365970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480822c5-fca6-4417-8346-641a99149912 service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:46.376336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1152/2311] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:29:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:29:53.725213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:53.802853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350369a-d3c9-47ef-a5c3-1ada9aecd033 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:53.807963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:53.808594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:53.808875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:53.809192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] PATH_INFO: `/v3/users/4a4c7d0ba47c452a8ab431c41c00ac04` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:53.810316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4c7d0ba47c452a8ab431c41c00ac04)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:53.812337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:53.967342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:53.973652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4c7d0ba47c452a8ab431c41c00ac04'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:53.981602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4c7d0ba47c452a8ab431c41c00ac04'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:54.044323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4c7d0ba47c452a8ab431c41c00ac04'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:54.044960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4c7d0ba47c452a8ab431c41c00ac04'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:54.064742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4c7d0ba47c452a8ab431c41c00ac04'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:54.074630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1160/2312] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:29:53 2022] DELETE /identity/v3/users/4a4c7d0ba47c452a8ab431c41c00ac04 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:29:54.079365 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:54.079914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:54.080242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:54.106107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:54.107652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:54.153115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:54.180244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1153/2313] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:29:54.187577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:54.189879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:54.265562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88d97-21b6-4798-b784-0b6a1a74c44c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:54.270591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:54.271310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:54.271610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:54.272061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:54.283995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:54.285742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:54.297887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1161/2314] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:54.848679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:54.854889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:54.865574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd765-de5f-4897-b365-dcaf6cfa6bee None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:54.870893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:54.871655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:54.872063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:54.872203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] PATH_INFO: `/v3/projects/67cecf7022af4316922c5173db3a3636` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:54.873385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=67cecf7022af4316922c5173db3a3636)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:54.875412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:55.014286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67cecf7022af4316922c5173db3a3636', 'request_id': 'req-f5ea011f-39f5-42e1-9e74-61464d7919cf'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:55.020606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67cecf7022af4316922c5173db3a3636', 'request_id': 'req-f5ea011f-39f5-42e1-9e74-61464d7919cf'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:55.028611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67cecf7022af4316922c5173db3a3636', 'request_id': 'req-f5ea011f-39f5-42e1-9e74-61464d7919cf'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:55.096737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] The token cache is being invalidate because project 67cecf7022af4316922c5173db3a3636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:29:55.097478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:55.097615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5ea011f-39f5-42e1-9e74-61464d7919cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:29:55.099370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1154/2315] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:29:54 2022] DELETE /identity/v3/projects/67cecf7022af4316922c5173db3a3636 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:29:59.054342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:59.063984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc1f1d-1078-4c31-a03a-4ad52fd05adc service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:59.115837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:59.116444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:59.116725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:59.117024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:59.118768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:59.186062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:59.187700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:59.198104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1162/2316] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:05.093128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:09.194134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:09.315884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91d30-a20c-4e7e-933b-bbf2d581f039 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:09.316327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91d30-a20c-4e7e-933b-bbf2d581f039 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:09.316431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91d30-a20c-4e7e-933b-bbf2d581f039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:09.335310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f5b91d30-a20c-4e7e-933b-bbf2d581f039 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:09.336504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f5b91d30-a20c-4e7e-933b-bbf2d581f039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:09.390325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1155/2317] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 10 13:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:09.402825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:09.404401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:09.472732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63f8ce-48a1-41bf-8f45-310d0991fe14 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:09.477706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:09.478317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:09.478596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:09.478866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:09.486910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:09.488510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:09.496881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1163/2318] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:30:19.150453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:19.150910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:19.150998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:19.173054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] MFA Rules not processed for user `0ace2374550e40ce812407406c262ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:19.174301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:19.233495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1156/2319] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:30:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:19.266886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:19.268297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:19.325710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0f7b3-e109-4706-8dcd-db2a592a855a service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:19.329341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:19.329943 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:19.330254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:19.330523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:19.332221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:19.400533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:19.401994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:19.413255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1164/2320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:20.189497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:20.191417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:20.204631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100ceea-70af-423d-b95f-0e945bca9cf4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:20.208400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:20.209208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:20.209489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:20.209780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:20.211480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:20.229179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:20.231221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:20.243385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1157/2321] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:29.408305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:29.430680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:29.432079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:29.451034 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7228c-e58a-4d31-9ec0-a83d12f063a3 service glance] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:29.456778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:29.457454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:29.457709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:29.457983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:29.460071 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:29.471328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:29.472906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:29.481885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1165/2322] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:30.216395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:30.227206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c978-1c2e-4a10-9b05-09c6405c806d service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:30.232609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:30.233265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:30.233574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:30.233843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:30.235528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:30.294728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:30.296308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:30.305414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1158/2323] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:39.477665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:40.301179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=84 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:41.160472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:41.162046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:41.177213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d396-e16d-4fcf-82c1-012f43c557b8 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:41.192807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:41.193904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:41.194190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:41.194463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:41.196336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:41.273733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:41.275526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:41.286061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1166/2324] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:48.414797 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:48.426542 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c7d2-43fe-442b-9751-555cfae7077e service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:48.432119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563083b-06f8-4a01-913e-00228181131b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:48.432712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563083b-06f8-4a01-913e-00228181131b service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:48.433025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563083b-06f8-4a01-913e-00228181131b service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:48.433302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563083b-06f8-4a01-913e-00228181131b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:48.435279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9563083b-06f8-4a01-913e-00228181131b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:48.447140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563083b-06f8-4a01-913e-00228181131b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:48.448705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563083b-06f8-4a01-913e-00228181131b service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:48.459202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1159/2325] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:49.996263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:49.997813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:50.009341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee9e2-ac2a-4d75-9ff8-d294dbbe8515 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:50.013020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:50.013676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:50.013952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:50.014223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:50.017111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:50.080853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:50.082245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:50.092511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1167/2326] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:53.827195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563083b-06f8-4a01-913e-00228181131b service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:53.828869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9563083b-06f8-4a01-913e-00228181131b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.905259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563083b-06f8-4a01-913e-00228181131b service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:53.910661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:53.911265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.911548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.911821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.913594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.926677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:53.928624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:53.939934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1160/2327] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:53.979078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:53.979078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.992556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dcd04-eb61-404b-95f1-cd4634173b7d service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:53.998393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.001466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.001466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.001466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] PATH_INFO: `/v3/users/4dfd3ee79da943b189824630b5c393ee` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.001783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfd3ee79da943b189824630b5c393ee)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.003995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:54.175667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfd3ee79da943b189824630b5c393ee'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.188969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfd3ee79da943b189824630b5c393ee'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.265980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfd3ee79da943b189824630b5c393ee'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.266591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfd3ee79da943b189824630b5c393ee'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.281080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b11fac38-37bf-4b5d-b6b5-5bd90d387dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfd3ee79da943b189824630b5c393ee'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.285584 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1168/2328] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:30:53 2022] DELETE /identity/v3/users/4dfd3ee79da943b189824630b5c393ee => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:54.541677 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:54.552707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05772270-1917-45e1-82ba-c24fc53b4438 service placement] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:54.611818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.612457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.612738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.613034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] PATH_INFO: `/v3/projects/c0c9fae66a6f4ac9a0c1b0629e597906` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.614167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c9fae66a6f4ac9a0c1b0629e597906)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.616032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:54.712296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c9fae66a6f4ac9a0c1b0629e597906', 'request_id': 'req-437b98cf-bb66-418d-8972-dd8925c38d8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.719087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c9fae66a6f4ac9a0c1b0629e597906', 'request_id': 'req-437b98cf-bb66-418d-8972-dd8925c38d8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.726223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c9fae66a6f4ac9a0c1b0629e597906', 'request_id': 'req-437b98cf-bb66-418d-8972-dd8925c38d8e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.804433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] The token cache is being invalidate because project c0c9fae66a6f4ac9a0c1b0629e597906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:30:54.804433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.804433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-437b98cf-bb66-418d-8972-dd8925c38d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.804433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1161/2329] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:30:54 2022] DELETE /identity/v3/projects/c0c9fae66a6f4ac9a0c1b0629e597906 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:54.805992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.806346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.806694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.817023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:54.828963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:54.830272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:54.893510 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1169/2330] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:54.901125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.901611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.901705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.916576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:54.923071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:54.923974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:54.932117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1162/2331] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:30:54.937833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:54.947363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee90f2e-076d-4a35-a4a6-5fae4410385b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:54.952623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.965985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.966773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.966773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.967259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:30:54.967567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.969225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:54.985079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1170/2332] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:30:54.992389 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.001779 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bbf26-ee61-4d1b-b5e9-bee25d42978b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.006522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.007123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.007406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.007681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.008307 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.010248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.114719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1163/2333] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:30:54 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:30:55.118985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.129303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3020d-5436-41ad-8557-a8d9e1ea6285 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.193706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.193706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.193706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.193706 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.194152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.194152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.272493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1171/2334] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:30:55 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:30:55.276646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.289269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047f09f-b3ab-4830-9fc8-a825eb2876ee admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.301718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.302485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.302919 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.303476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.304281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.305955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.318082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1164/2335] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:30:55.322211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.333361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde56b-413e-4990-8533-8dabc03bdca7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.338181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa99ac-bf6d-4c7d-b934-a79441122b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.338812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa99ac-bf6d-4c7d-b934-a79441122b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.339091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa99ac-bf6d-4c7d-b934-a79441122b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.339359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa99ac-bf6d-4c7d-b934-a79441122b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.340074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa99ac-bf6d-4c7d-b934-a79441122b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.342095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa99ac-bf6d-4c7d-b934-a79441122b54 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.354464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1172/2336] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:30:55.357933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.367950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844350c-4de4-4e3e-b207-0c36f5cdf9d1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.372732 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.373475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.373767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.374119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] PATH_INFO: `/v3/projects/2530c9b8abc645caa479fa487f76e9c9/users/ce236e8ea2ce40cfaa26b185c686f328/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.390493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2530c9b8abc645caa479fa487f76e9c9, user_id=ce236e8ea2ce40cfaa26b185c686f328, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.393260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.500501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1165/2337] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:30:55 2022] PUT /identity/v3/projects/2530c9b8abc645caa479fa487f76e9c9/users/ce236e8ea2ce40cfaa26b185c686f328/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:55.510682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.510974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.511259 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.541737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] MFA Rules not processed for user `ce236e8ea2ce40cfaa26b185c686f328`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:55.542988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:55.607992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:55.611675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1173/2338] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:30:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:55.616148 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.689147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88cb6-9b4d-487a-89c2-a31186d54854 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.691795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.692411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.692686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.693041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.693671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.695557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.794073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1166/2339] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:30:55 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:30:55.794439 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.805401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fa1c-2db9-464f-804c-011e10c88ea1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.871821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.871821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.871821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.871821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.872461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.872461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.952744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1174/2340] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:30:55 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:30:55.956691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:55.967622 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc607d6-13ee-4a91-bdb0-aa58a6c40428 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:55.972238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:55.975009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:55.975009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:55.975009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:55.975009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:55.975818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.986401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1167/2341] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:30:55.990684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:56.001295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7121a6-7d7d-4d3a-9800-7781e9e78bd7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:56.005883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:56.006525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.006805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.007119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] PATH_INFO: `/v3/projects/98295e7b02444f00bf103b24189eacf2/users/9f2a1853e7844661b7926fbb64b3264a/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.022697 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98295e7b02444f00bf103b24189eacf2, user_id=9f2a1853e7844661b7926fbb64b3264a, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:56.025349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:56.095092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1175/2342] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:30:55 2022] PUT /identity/v3/projects/98295e7b02444f00bf103b24189eacf2/users/9f2a1853e7844661b7926fbb64b3264a/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:30:56.099133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:56.110059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cfeb2-054f-4264-8565-6063e713c946 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:56.166749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30686806-4bce-4002-a1d1-d51d162f9c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:56.167250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30686806-4bce-4002-a1d1-d51d162f9c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.167528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30686806-4bce-4002-a1d1-d51d162f9c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.167799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30686806-4bce-4002-a1d1-d51d162f9c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.168428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30686806-4bce-4002-a1d1-d51d162f9c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:56.170428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30686806-4bce-4002-a1d1-d51d162f9c87 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:56.179838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1168/2343] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:30:56.183957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:56.194264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ece4-10bd-40c3-9a86-b1d5b6004317 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:56.198819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:56.199457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.199736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.200008 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] PATH_INFO: `/v3/projects/98295e7b02444f00bf103b24189eacf2/users/9f2a1853e7844661b7926fbb64b3264a/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.202603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98295e7b02444f00bf103b24189eacf2, user_id=9f2a1853e7844661b7926fbb64b3264a, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:56.205344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:56.215762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1176/2344] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:30:56 2022] PUT /identity/v3/projects/98295e7b02444f00bf103b24189eacf2/users/9f2a1853e7844661b7926fbb64b3264a/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:56.227533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.227827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.228109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.254790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:56.257763 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] MFA Rules not processed for user `9f2a1853e7844661b7926fbb64b3264a`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:56.258986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:56.336762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1169/2345] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:56.349015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:56.350715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:56.410308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc56d6-e23b-4f70-8a06-70acd935f040 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:56.414011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394bcde-aeff-4445-81c8-9985d67bcdf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:56.414612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394bcde-aeff-4445-81c8-9985d67bcdf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.414890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394bcde-aeff-4445-81c8-9985d67bcdf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.415162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394bcde-aeff-4445-81c8-9985d67bcdf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.424046 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394bcde-aeff-4445-81c8-9985d67bcdf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:56.425436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394bcde-aeff-4445-81c8-9985d67bcdf1 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:56.434567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1177/2346] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:56.650289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:56.661469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ddade-7c0e-480f-9c36-e962781b8a18 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:56.665149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf97745-47c9-464f-944c-ec73da53f7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:56.665773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf97745-47c9-464f-944c-ec73da53f7fe service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.666050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf97745-47c9-464f-944c-ec73da53f7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.666324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf97745-47c9-464f-944c-ec73da53f7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.725381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf97745-47c9-464f-944c-ec73da53f7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:56.726722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf97745-47c9-464f-944c-ec73da53f7fe service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:56.735730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1170/2347] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:06.430529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:06.731500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=56 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:08.301655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:08.302173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:08.302173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:08.321903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] MFA Rules not processed for user `c9577bc4c4c546ca9fc220ebd3a16346`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:08.323111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:08.385281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1178/2348] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:08.611290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:08.611828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:08.611930 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:08.638181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] MFA Rules not processed for user `1945fed03f73472db818c4e5ca015eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:08.638181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:08.658246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:08.715055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1171/2349] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:08.736525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:08.737881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:08.744258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2df71-441c-4fea-8aec-56ec411bc013 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:08.749853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:08.751581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:08.751581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:08.751581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] PATH_INFO: `/v3/users/ce236e8ea2ce40cfaa26b185c686f328` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:08.752231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce236e8ea2ce40cfaa26b185c686f328)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:08.754205 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:08.804700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eb237-e3eb-42ea-a372-cc6773141503 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:08.808557 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:08.814623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:08.814623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:08.814623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:08.814623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:08.821667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:08.823115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:08.832308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1172/2350] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:08.961371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce236e8ea2ce40cfaa26b185c686f328'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:08.971672 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce236e8ea2ce40cfaa26b185c686f328'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.043080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce236e8ea2ce40cfaa26b185c686f328'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.044610 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce236e8ea2ce40cfaa26b185c686f328'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.056742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:09.058163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.060955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0afd2446-689c-4622-905b-8c8ebbd85912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce236e8ea2ce40cfaa26b185c686f328'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.071052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33cd2-5a81-487a-9751-ad3f84c898cd service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:09.074427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1179/2351] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:08 2022] DELETE /identity/v3/users/ce236e8ea2ce40cfaa26b185c686f328 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:09.075906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.076282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.078220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.101123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:09.102691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.124578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:09.126595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:09.127225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.127508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.127786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.129545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.139562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:09.141206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:09.149514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1173/2352] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:09.165952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1180/2353] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:09.176060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:09.177497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.188623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5534-7bdf-4176-8829-10ee6e5960b3 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:09.192414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:09.193016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.193311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.193586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.204510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:09.207311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:09.218951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1174/2354] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:09.271293 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:09.272692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.366810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450724c7-7f9d-4f71-a316-9017e27875e3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:09.372324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:09.373079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.374052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.374327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.376118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.387734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:09.389413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:09.399572 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1181/2355] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:09.733615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:09.735051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.752818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee510de-e0eb-418c-81f2-b2db2313de03 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:09.758334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:09.761722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.761722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.761722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] PATH_INFO: `/v3/projects/2530c9b8abc645caa479fa487f76e9c9` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.761722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2530c9b8abc645caa479fa487f76e9c9)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:09.762666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:09.860592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:09.871987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2530c9b8abc645caa479fa487f76e9c9', 'request_id': 'req-b1be9805-01a1-42ff-870d-54990cd81822'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.878175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2530c9b8abc645caa479fa487f76e9c9', 'request_id': 'req-b1be9805-01a1-42ff-870d-54990cd81822'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.885224 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2530c9b8abc645caa479fa487f76e9c9', 'request_id': 'req-b1be9805-01a1-42ff-870d-54990cd81822'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.996702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] The token cache is being invalidate because project 2530c9b8abc645caa479fa487f76e9c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:09.997016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.997654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:09.999935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1175/2356] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:09 2022] DELETE /identity/v3/projects/2530c9b8abc645caa479fa487f76e9c9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:10.005127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:10.016786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91a9a5-77ea-40d0-afa8-06cfdb2056a1 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:10.084963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:10.087201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:10.087201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:10.087201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] PATH_INFO: `/v3/users/9f2a1853e7844661b7926fbb64b3264a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:10.087622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f2a1853e7844661b7926fbb64b3264a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:10.090563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:10.319526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f2a1853e7844661b7926fbb64b3264a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:10.326408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f2a1853e7844661b7926fbb64b3264a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:10.337425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:10.437509 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f2a1853e7844661b7926fbb64b3264a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:10.438107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f2a1853e7844661b7926fbb64b3264a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:10.450478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5182055-be9f-4057-aafb-998f1bb64e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f2a1853e7844661b7926fbb64b3264a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:10.456323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1182/2357] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:10 2022] DELETE /identity/v3/users/9f2a1853e7844661b7926fbb64b3264a => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:10.982037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:10.984948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:11.005503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be9805-01a1-42ff-870d-54990cd81822 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.066939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.067714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.067995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.068272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] PATH_INFO: `/v3/projects/98295e7b02444f00bf103b24189eacf2` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.069414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98295e7b02444f00bf103b24189eacf2)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.071493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.185502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98295e7b02444f00bf103b24189eacf2', 'request_id': 'req-b3266812-e041-4958-b197-1347b9541117'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:11.191547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98295e7b02444f00bf103b24189eacf2', 'request_id': 'req-b3266812-e041-4958-b197-1347b9541117'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:11.197558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98295e7b02444f00bf103b24189eacf2', 'request_id': 'req-b3266812-e041-4958-b197-1347b9541117'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:11.259272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] The token cache is being invalidate because project 98295e7b02444f00bf103b24189eacf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:11.259493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:11.260138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b3266812-e041-4958-b197-1347b9541117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:11.261666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1176/2358] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:10 2022] DELETE /identity/v3/projects/98295e7b02444f00bf103b24189eacf2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:11.266740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.267029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.267307 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.286950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:11.288192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:11.346780 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1183/2359] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:11.354421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.354716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.355000 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.376442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:11.377690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:11.386183 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1177/2360] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:11.390923 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:11.400319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74046c-6c9d-4749-bf9a-c1bc2193191c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.405180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.405846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.406127 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.406404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.407059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:31:11.407366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.409018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.424573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1184/2361] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:31:11.428963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:11.439129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbe18e-7f36-48b7-a272-61bc411110c4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.444099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.444709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.448871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.448871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.448871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.448871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.553474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1178/2362] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:11 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:11.560632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:11.571332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f8124-7549-416c-8787-c396f4c7ccff admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.632491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.633145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.633424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.633694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.634328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.636331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.741800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1185/2363] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:11 2022] POST /identity/v3/users => generated 282 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:11.745549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:11.759351 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112e0b7-dc91-448a-9220-41889a90f5a1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.766083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.766927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.767422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.767700 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.768435 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.770242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.781377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1179/2364] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:11.786359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:11.800132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c995-778f-4b5e-bfba-0b793f37fa7e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.822173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4027-71d0-43ed-9802-8df90106becb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.822877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4027-71d0-43ed-9802-8df90106becb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.823620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4027-71d0-43ed-9802-8df90106becb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.823620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4027-71d0-43ed-9802-8df90106becb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.824367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4027-71d0-43ed-9802-8df90106becb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.826060 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4027-71d0-43ed-9802-8df90106becb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.847707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1186/2365] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:11.852192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:11.863609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be53fc-8e48-44c1-923b-93f92b77720b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.873719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.875203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.875666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.876053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] PATH_INFO: `/v3/projects/af83afb389a74ff5919f3d7630eb03a6/users/805e52fbf3b6438795caa601a33bd6fa/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.897313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:11.905263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af83afb389a74ff5919f3d7630eb03a6, user_id=805e52fbf3b6438795caa601a33bd6fa, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.908071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.032647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1180/2366] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:11 2022] PUT /identity/v3/projects/af83afb389a74ff5919f3d7630eb03a6/users/805e52fbf3b6438795caa601a33bd6fa/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:12.044861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.045177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.045449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.074354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] MFA Rules not processed for user `805e52fbf3b6438795caa601a33bd6fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:12.075638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:12.107404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:12.147171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1187/2367] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:12.151844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.231046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb1283-1b15-451f-876c-0e930111e6f9 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:12.236492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:12.237129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.237409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.237682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.238305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:12.240337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.341784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1181/2368] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:12 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:12.346019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.357212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be7e73-81a8-495e-9c11-5cd65d584ca1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:12.444604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:12.446035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.446035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.446353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.447146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:12.449723 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.558716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1188/2369] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:12 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:12.565648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.580707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5246c-0349-4c56-8c60-10bcc1422ccc admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:12.586254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:12.586914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.587197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.596150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.596150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:12.596150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.616302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1182/2370] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:12.620815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.633292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fadfd1-2f17-4e36-8bf6-41722cb50afb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:12.639352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:12.640299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.640959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.641425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] PATH_INFO: `/v3/projects/3c659b587d8340aa9ea852e8d37c169c/users/a51460ca510042119f1efad72f2941af/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.663322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c659b587d8340aa9ea852e8d37c169c, user_id=a51460ca510042119f1efad72f2941af, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:12.666468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.737013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1189/2371] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:12 2022] PUT /identity/v3/projects/3c659b587d8340aa9ea852e8d37c169c/users/a51460ca510042119f1efad72f2941af/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:12.742624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.754282 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eec8de-9756-4f9b-bf0b-835c71e2f996 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:12.813248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78b20e-4022-4358-9ed2-343b36c92393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:12.813884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78b20e-4022-4358-9ed2-343b36c92393 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.814169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78b20e-4022-4358-9ed2-343b36c92393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.814447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78b20e-4022-4358-9ed2-343b36c92393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.815095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78b20e-4022-4358-9ed2-343b36c92393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:12.816697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78b20e-4022-4358-9ed2-343b36c92393 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.828156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1183/2372] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:12.831999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.842611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7110c-94b4-4f2f-b7be-56fe57d71418 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:12.848048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:12.848716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.849018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.849324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] PATH_INFO: `/v3/projects/3c659b587d8340aa9ea852e8d37c169c/users/a51460ca510042119f1efad72f2941af/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.851217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c659b587d8340aa9ea852e8d37c169c, user_id=a51460ca510042119f1efad72f2941af, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:12.853895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:12.865918 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1190/2373] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:12 2022] PUT /identity/v3/projects/3c659b587d8340aa9ea852e8d37c169c/users/a51460ca510042119f1efad72f2941af/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:12.876111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.876403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.876674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.906428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] MFA Rules not processed for user `a51460ca510042119f1efad72f2941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:12.907744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:12.964932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:13.022407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1184/2374] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:13.034620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:13.036440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:13.103134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704fc6e-439c-40fb-9581-ddf806abc713 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:13.107120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:13.107736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:13.108015 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:13.108289 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:13.119230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:13.120709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:13.131699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1191/2375] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:13.146400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:13.148198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:13.217090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3348edb-983b-4811-9ccf-e1b5b2601338 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:13.221879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:13.222125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:13.222310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:13.222587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:13.224419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:13.257950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:13.259531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110508ce-fc6e-455a-8809-de2f23a919bb service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:13.272011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1185/2376] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:13.996888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:14.027783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:14.070623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d248-d33c-433d-8536-c4b09db2d9ee service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:14.075709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:14.076472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:14.076751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:14.077043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] PATH_INFO: `/v3/users/805e52fbf3b6438795caa601a33bd6fa` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:14.079171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805e52fbf3b6438795caa601a33bd6fa)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:14.081212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:14.302860 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805e52fbf3b6438795caa601a33bd6fa'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:14.310066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805e52fbf3b6438795caa601a33bd6fa'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:14.380010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805e52fbf3b6438795caa601a33bd6fa'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:14.380630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805e52fbf3b6438795caa601a33bd6fa'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:14.395098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805e52fbf3b6438795caa601a33bd6fa'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:14.401219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1192/2377] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:13 2022] DELETE /identity/v3/users/805e52fbf3b6438795caa601a33bd6fa => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:14.406150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:14.406788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:14.406879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:14.431813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:14.433095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:14.494609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1186/2378] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:14.505325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:14.506743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:14.568694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319413cb-8f85-4fff-981d-ad1387531d55 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:14.572375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:14.573013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:14.576520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:14.576520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:14.583752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:14.585238 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:14.595568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1193/2379] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:15.026872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:15.028338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:15.041826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78f86e-e303-4676-a07e-0e1c4128b4a4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:15.047738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:15.048385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:15.052968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:15.052968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] PATH_INFO: `/v3/projects/af83afb389a74ff5919f3d7630eb03a6` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:15.052968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af83afb389a74ff5919f3d7630eb03a6)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:15.052968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:15.205145 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af83afb389a74ff5919f3d7630eb03a6', 'request_id': 'req-55527766-2240-433e-96ba-940b112b7092'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.212510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af83afb389a74ff5919f3d7630eb03a6', 'request_id': 'req-55527766-2240-433e-96ba-940b112b7092'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.219388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af83afb389a74ff5919f3d7630eb03a6', 'request_id': 'req-55527766-2240-433e-96ba-940b112b7092'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.322412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] The token cache is being invalidate because project af83afb389a74ff5919f3d7630eb03a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:15.324111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.324929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.326686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1187/2380] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:15 2022] DELETE /identity/v3/projects/af83afb389a74ff5919f3d7630eb03a6 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:15.331298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:15.341441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9207-90f1-41af-8749-202122bb596c service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:15.391854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:15.407631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:15.408579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:15.408862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:15.409466 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] PATH_INFO: `/v3/users/a51460ca510042119f1efad72f2941af` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:15.410704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51460ca510042119f1efad72f2941af)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:15.412900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:15.604345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51460ca510042119f1efad72f2941af'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.612346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51460ca510042119f1efad72f2941af'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.677245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51460ca510042119f1efad72f2941af'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.678009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51460ca510042119f1efad72f2941af'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.691592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-205ad425-7895-4a74-bc8f-6eac300fb543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51460ca510042119f1efad72f2941af'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:15.697543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1194/2381] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:15 2022] DELETE /identity/v3/users/a51460ca510042119f1efad72f2941af => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:16.113356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.114872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:16.128256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55527766-2240-433e-96ba-940b112b7092 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.133367 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:16.188846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.189514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.189799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.190074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] PATH_INFO: `/v3/projects/3c659b587d8340aa9ea852e8d37c169c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.191550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c659b587d8340aa9ea852e8d37c169c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.193524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.312786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c659b587d8340aa9ea852e8d37c169c', 'request_id': 'req-30895797-497a-4cee-b6c3-b7210c372ced'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:16.318139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c659b587d8340aa9ea852e8d37c169c', 'request_id': 'req-30895797-497a-4cee-b6c3-b7210c372ced'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:16.324228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c659b587d8340aa9ea852e8d37c169c', 'request_id': 'req-30895797-497a-4cee-b6c3-b7210c372ced'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:16.412771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] The token cache is being invalidate because project 3c659b587d8340aa9ea852e8d37c169c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:16.412987 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:16.413631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-30895797-497a-4cee-b6c3-b7210c372ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:16.415154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1188/2382] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:16 2022] DELETE /identity/v3/projects/3c659b587d8340aa9ea852e8d37c169c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:16.419956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.420422 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.420516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.440398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:16.441637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:16.502812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1195/2383] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:16.510428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.510719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.511016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.532479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:16.533734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:16.541982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1189/2384] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:16.547003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.556613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281701d-1c5c-4f87-885c-3c1a51d98efe None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.561304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.562058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.562339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.562620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.563308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:31:16.563617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.565241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.580359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1196/2385] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:31:16.581570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.590813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58bb8d-17ee-494e-a3ea-ea459200b7f2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.596510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.597154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.597436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.597710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.598348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.600291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.679110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1190/2386] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:16 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:31:16.683097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.694247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce8899-f2d6-4db6-878a-9ee0953bfd20 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.746389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:16.756116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.756910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.757310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.757682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.758488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.760768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.865515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1197/2387] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:16 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:16.869598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.880941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04f042-c0f4-41d1-ac6b-66447e9db760 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.885806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.886423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.886703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.886974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.887617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.889233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.899969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1191/2388] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:31:16.903979 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.914586 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c26a3-d3df-4b61-a7cc-fd780d726daa admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.919425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128ada6-a6ff-4398-951f-25e24a710c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.920023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128ada6-a6ff-4398-951f-25e24a710c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.920322 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128ada6-a6ff-4398-951f-25e24a710c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.920607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128ada6-a6ff-4398-951f-25e24a710c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.922443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128ada6-a6ff-4398-951f-25e24a710c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.924325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128ada6-a6ff-4398-951f-25e24a710c15 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.935087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1198/2389] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:16.939754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:16.949988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb7c5-acde-4f9f-b9d6-f7bd58068d05 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:16.954619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:16.955301 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:16.955583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:16.955856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] PATH_INFO: `/v3/projects/665ff2ec2252498d909a165ae8ebd207/users/0919f96590e54b4586c1d485e432a84b/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:16.970423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665ff2ec2252498d909a165ae8ebd207, user_id=0919f96590e54b4586c1d485e432a84b, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.973118 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.076915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1192/2390] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:16 2022] PUT /identity/v3/projects/665ff2ec2252498d909a165ae8ebd207/users/0919f96590e54b4586c1d485e432a84b/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:17.088495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.088789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.089100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.118680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] MFA Rules not processed for user `0919f96590e54b4586c1d485e432a84b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:17.119917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:17.183954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1199/2391] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:17.189062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:17.227391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:17.249993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab99b8-5b08-46bd-a92c-5079f7c02978 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:17.254654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:17.255258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.255536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.255804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.256438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:17.258387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.392548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1193/2392] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:17 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:17.396541 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:17.408062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0533aa-3369-4246-a55d-e67c2841f226 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:17.464512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:17.465122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.465339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.465612 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.466242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:17.468305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.573691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1200/2393] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:17 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:17.578315 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:17.588655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aa8ad-fe5a-4006-a67c-a2f86b4c5b1c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:17.593933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:17.594610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.594892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.595165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.595963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:17.597661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.608938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1194/2394] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:17.613663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:17.630577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aceb7b-bcc8-4711-a8d9-5c2d24ba50f1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:17.637898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:17.638763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.639275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.639739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] PATH_INFO: `/v3/projects/af87655142f444b48dd501286dc8a13e/users/85258da97b004a37904ad26111523a25/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.658892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af87655142f444b48dd501286dc8a13e, user_id=85258da97b004a37904ad26111523a25, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:17.662649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.775029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1201/2395] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:17 2022] PUT /identity/v3/projects/af87655142f444b48dd501286dc8a13e/users/85258da97b004a37904ad26111523a25/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:17.779054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:17.791728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6f9d-ef76-4cb9-b481-2b76aa572bfd admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:17.907178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3bf87-9e15-48b1-b145-03b6686c1d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:17.907955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3bf87-9e15-48b1-b145-03b6686c1d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.908306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3bf87-9e15-48b1-b145-03b6686c1d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.908641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3bf87-9e15-48b1-b145-03b6686c1d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.909868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3bf87-9e15-48b1-b145-03b6686c1d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:17.912090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3bf87-9e15-48b1-b145-03b6686c1d2f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.927171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1195/2396] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:17.930100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:17.942588 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03762-5ce3-42a6-9c72-2786afc95ec3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:17.948083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:17.948844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.949161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.949436 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] PATH_INFO: `/v3/projects/af87655142f444b48dd501286dc8a13e/users/85258da97b004a37904ad26111523a25/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:17.951393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af87655142f444b48dd501286dc8a13e, user_id=85258da97b004a37904ad26111523a25, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:17.955073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:17.967459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1202/2397] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:17 2022] PUT /identity/v3/projects/af87655142f444b48dd501286dc8a13e/users/85258da97b004a37904ad26111523a25/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:17.979514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:17.979806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:17.980083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:18.027319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] MFA Rules not processed for user `85258da97b004a37904ad26111523a25`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:18.027319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:18.098881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1196/2398] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:18.111454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:18.113292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:18.129068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:18.175195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa524f52-b976-4dcf-a177-d83ff1345c56 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:18.179372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:18.179989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:18.180275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:18.180548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:18.190887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:18.192703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:18.202341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1203/2399] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:18.428342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:18.437940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f701d-9dfa-49a0-aec1-12a69a8daff7 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:18.441409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:18.441999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:18.442277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:18.442548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:18.471978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:18.498885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:18.501081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:18.510496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1197/2400] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:19.975403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:19.976956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:20.070387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09ef39-844a-44d5-853f-5b09b9a3d622 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:20.079002 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:20.079687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:20.080045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:20.080320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:20.082137 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:20.096578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:20.098261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2656e94-94cf-4519-99d1-188a5cfa904e service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:20.110474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1204/2401] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:22.210225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:22.296925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569fa9-6093-487e-a417-8c202dae0813 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:22.302236 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:22.302893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:22.303173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:22.303441 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] PATH_INFO: `/v3/users/0919f96590e54b4586c1d485e432a84b` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:22.304670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0919f96590e54b4586c1d485e432a84b)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:22.306773 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:22.514027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0919f96590e54b4586c1d485e432a84b'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:22.585667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0919f96590e54b4586c1d485e432a84b'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:22.594071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0919f96590e54b4586c1d485e432a84b'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:22.602597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0919f96590e54b4586c1d485e432a84b'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:22.608850 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0919f96590e54b4586c1d485e432a84b'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:22.615512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1198/2402] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:22 2022] DELETE /identity/v3/users/0919f96590e54b4586c1d485e432a84b => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:22.620519 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:22.627248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:22.627248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:22.642265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:22.643774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:22.721998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1205/2403] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:22.732464 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:22.734771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:22.794512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:22.799268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efab66-a3b0-49eb-b77e-89a50b59e2bd admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:22.803336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:22.803954 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:22.804237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:22.804513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:22.814683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:22.816127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:22.827539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1199/2404] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:23.317974 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:23.319429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:23.331499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03712f-8fe6-4487-aac1-49c510ec1a4b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:23.336468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:23.337156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:23.337446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:23.337720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] PATH_INFO: `/v3/projects/665ff2ec2252498d909a165ae8ebd207` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:23.338855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=665ff2ec2252498d909a165ae8ebd207)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:23.340758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:23.438875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:23.448373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665ff2ec2252498d909a165ae8ebd207', 'request_id': 'req-5e32ddec-c06d-4c70-9892-09654eecc4bc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.457713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665ff2ec2252498d909a165ae8ebd207', 'request_id': 'req-5e32ddec-c06d-4c70-9892-09654eecc4bc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.464432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665ff2ec2252498d909a165ae8ebd207', 'request_id': 'req-5e32ddec-c06d-4c70-9892-09654eecc4bc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.534400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] The token cache is being invalidate because project 665ff2ec2252498d909a165ae8ebd207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:23.534689 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.535325 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.536823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1206/2405] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:23 2022] DELETE /identity/v3/projects/665ff2ec2252498d909a165ae8ebd207 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:23.540822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:23.550958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff061bc-becf-4d43-9633-3dbd60a74beb service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:23.611526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:23.612173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:23.612478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:23.612752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] PATH_INFO: `/v3/users/85258da97b004a37904ad26111523a25` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:23.613924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85258da97b004a37904ad26111523a25)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:23.615916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:23.824339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85258da97b004a37904ad26111523a25'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.899109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85258da97b004a37904ad26111523a25'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.909321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85258da97b004a37904ad26111523a25'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.910013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85258da97b004a37904ad26111523a25'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.922671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85258da97b004a37904ad26111523a25'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:23.928735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1200/2406] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:23 2022] DELETE /identity/v3/users/85258da97b004a37904ad26111523a25 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:24.149718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:24.152449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:24.255566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32ddec-c06d-4c70-9892-09654eecc4bc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:24.363986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:24.364550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:24.364834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:24.365193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:24.367039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:24.380828 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:24.382480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9f5ea-d7f9-4899-adfc-09bfc33cee4f service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:24.394198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:24.396153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:24.402802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1207/2407] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:24.517587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16ff8-43a4-41d1-8c63-dc9b7cf61aa5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:24.523507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:24.525394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:24.525394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:24.525394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] PATH_INFO: `/v3/projects/af87655142f444b48dd501286dc8a13e` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:24.529651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af87655142f444b48dd501286dc8a13e)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:24.532586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:24.546466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:24.674758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af87655142f444b48dd501286dc8a13e', 'request_id': 'req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:24.683276 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af87655142f444b48dd501286dc8a13e', 'request_id': 'req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:24.691362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af87655142f444b48dd501286dc8a13e', 'request_id': 'req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:24.753172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] The token cache is being invalidate because project af87655142f444b48dd501286dc8a13e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:24.753616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:24.754447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4bb38f05-ca7a-4b0e-9344-cb291f06bbb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:24.759586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1201/2408] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:24 2022] DELETE /identity/v3/projects/af87655142f444b48dd501286dc8a13e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:24.761627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:24.761915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:24.762191 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:24.784955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:24.786215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:24.840820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:24.858795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1208/2409] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:24.867090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:24.867438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:24.867664 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:24.891603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:24.892961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:24.902125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1202/2410] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:24.906843 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:24.917592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651c5e1-ceab-4ccc-b085-151653efa756 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:24.924620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:24.925762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:24.926691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:24.927258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:24.928094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:31:24.928547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:24.930545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:24.947844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1209/2411] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:31:24.949262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:24.961189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b5fa9-bfa1-49c6-87ac-ec31b6a56fef None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:24.966592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:24.967217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:24.967558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:24.967939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:24.968594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:24.970731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:25.064813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1203/2412] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:24 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:25.066186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:25.080055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22bda8-d5f2-41ff-a305-f60ba5d3e3d6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:25.165442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:25.166004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.166108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.166346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.167114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:25.169545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:25.277922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1210/2413] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:25 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:25.283018 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:25.299513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf6d6-eb71-436b-8d4c-bb8b062f2842 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:25.305320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:25.306305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.306305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.306555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.307125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:25.308740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:25.322998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1204/2414] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:25.327219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:25.335533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c93e-d72f-41be-b03f-c681a5d76797 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:25.340338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8c11-ac0c-4fe1-9e36-a8c43c9a979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:25.340957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8c11-ac0c-4fe1-9e36-a8c43c9a979b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.341277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8c11-ac0c-4fe1-9e36-a8c43c9a979b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.341554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8c11-ac0c-4fe1-9e36-a8c43c9a979b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.342210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8c11-ac0c-4fe1-9e36-a8c43c9a979b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:25.344702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8c11-ac0c-4fe1-9e36-a8c43c9a979b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:25.360392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1211/2415] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:25.361734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:25.372645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b710-fa65-4ab5-af14-e5535b5cd7fc admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:25.380027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:25.380027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.380566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.380662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] PATH_INFO: `/v3/projects/ff8e03fce3d846229c345f365d19549e/users/0554ed6cc15d4ce298c6accc3a0334eb/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.397637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8e03fce3d846229c345f365d19549e, user_id=0554ed6cc15d4ce298c6accc3a0334eb, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:25.400846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:25.494131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.494408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.494713 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1205/2416] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:25 2022] PUT /identity/v3/projects/ff8e03fce3d846229c345f365d19549e/users/0554ed6cc15d4ce298c6accc3a0334eb/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:25.494863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.547470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] MFA Rules not processed for user `0554ed6cc15d4ce298c6accc3a0334eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:25.549030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:25.631355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1212/2417] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:25.635974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:25.716094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bace6ef-2432-46ee-9548-0d2de3686bca admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:25.722046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:25.722793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.723195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.724028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.724317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:25.726446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:25.840906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1206/2418] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:25 2022] POST /identity/v3/projects => generated 369 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:25.842304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:25.854222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707b91-f373-4712-ad4c-993fb4016181 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:25.901143 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:25.917324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:25.917941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:25.918251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:25.918525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:25.919156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:25.921910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:26.032676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1213/2419] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:25 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:26.037140 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:26.047909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18285-9383-492a-a362-c9c2f5876bb5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:26.057037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:26.057037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:26.057037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:26.057037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:26.057503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:26.057503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:26.071205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1207/2420] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:26.072606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:26.083540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac10d0-2183-4877-9093-bb117d878d54 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:26.088135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:26.088781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:26.089094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:26.089365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] PATH_INFO: `/v3/projects/4aa613bc5a684d408d7e8b545c1c3a76/users/3a97445f95204063b4c13c61d99271d5/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:26.104532 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa613bc5a684d408d7e8b545c1c3a76, user_id=3a97445f95204063b4c13c61d99271d5, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:26.107198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:26.191988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1214/2421] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:26 2022] PUT /identity/v3/projects/4aa613bc5a684d408d7e8b545c1c3a76/users/3a97445f95204063b4c13c61d99271d5/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:26.195881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:26.206680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202424de-bc01-4343-86f9-ea4314036ee0 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:26.214714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:26.265748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be188459-80df-4dcb-93fc-376405c57710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:26.266353 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be188459-80df-4dcb-93fc-376405c57710 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:26.266630 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be188459-80df-4dcb-93fc-376405c57710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:26.266900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be188459-80df-4dcb-93fc-376405c57710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:26.267543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be188459-80df-4dcb-93fc-376405c57710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:26.269191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be188459-80df-4dcb-93fc-376405c57710 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:26.278654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1208/2422] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:26.283536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:26.294221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bd5ec-fa2c-46ea-a2ff-382de3260761 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:26.299566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:26.300314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:26.300594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:26.300866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] PATH_INFO: `/v3/projects/4aa613bc5a684d408d7e8b545c1c3a76/users/3a97445f95204063b4c13c61d99271d5/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:26.302787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa613bc5a684d408d7e8b545c1c3a76, user_id=3a97445f95204063b4c13c61d99271d5, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:26.305482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:26.317186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1215/2423] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:26 2022] PUT /identity/v3/projects/4aa613bc5a684d408d7e8b545c1c3a76/users/3a97445f95204063b4c13c61d99271d5/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:26.328170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:26.328771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:26.329308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:26.357764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] MFA Rules not processed for user `3a97445f95204063b4c13c61d99271d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:26.359522 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:26.432461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1209/2424] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:26.441230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:26.445866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:26.508170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdd96d-5556-4a74-95be-82032ad8b7d1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:26.511817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496678d-9383-4a58-a064-87e2be66f4ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:26.512418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496678d-9383-4a58-a064-87e2be66f4ef service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:26.512693 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496678d-9383-4a58-a064-87e2be66f4ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:26.512963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496678d-9383-4a58-a064-87e2be66f4ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:26.522349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496678d-9383-4a58-a064-87e2be66f4ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:26.523767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496678d-9383-4a58-a064-87e2be66f4ef service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:26.534743 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1216/2425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:29.569550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:29.631344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8221-ff61-41d7-9a38-7a38a2a194cd None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:29.635834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:29.636492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:29.636768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:29.637059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] PATH_INFO: `/v3/users/0554ed6cc15d4ce298c6accc3a0334eb` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:29.638188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0554ed6cc15d4ce298c6accc3a0334eb)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:29.640711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:29.813807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:29.822782 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0554ed6cc15d4ce298c6accc3a0334eb'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:29.899249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0554ed6cc15d4ce298c6accc3a0334eb'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:29.907009 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0554ed6cc15d4ce298c6accc3a0334eb'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:29.907666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0554ed6cc15d4ce298c6accc3a0334eb'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:29.920429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0554ed6cc15d4ce298c6accc3a0334eb'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:29.926418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1210/2426] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:29 2022] DELETE /identity/v3/users/0554ed6cc15d4ce298c6accc3a0334eb => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:29.930795 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:29.931251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:29.931368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:29.950511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:29.952303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:30.011687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1217/2427] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:30.022750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:30.024195 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:30.082749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65439c9f-ad59-49e7-a708-20202f84392a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:30.086361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:30.087021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:30.087297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:30.087566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:30.096407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:30.097914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:30.108624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1211/2428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:30.522694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:30.522694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:30.534871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fa0ae-4b8e-4073-a4b3-2716355d1972 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:30.540410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:30.541103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:30.541387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:30.545352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] PATH_INFO: `/v3/projects/ff8e03fce3d846229c345f365d19549e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:30.545352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8e03fce3d846229c345f365d19549e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:30.545352 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:30.640750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:30.650661 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8e03fce3d846229c345f365d19549e', 'request_id': 'req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:30.656459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8e03fce3d846229c345f365d19549e', 'request_id': 'req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:30.663283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8e03fce3d846229c345f365d19549e', 'request_id': 'req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:30.750838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] The token cache is being invalidate because project ff8e03fce3d846229c345f365d19549e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:30.751128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:30.751758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:30.753290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1218/2429] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:30 2022] DELETE /identity/v3/projects/ff8e03fce3d846229c345f365d19549e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:30.757350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:30.767802 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6963230-747c-4b4b-8dcc-b3c8be6a7be5 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:30.829440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:30.830090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:30.830421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:30.830648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] PATH_INFO: `/v3/users/3a97445f95204063b4c13c61d99271d5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:30.831796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a97445f95204063b4c13c61d99271d5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:30.833833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:31.035495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a97445f95204063b4c13c61d99271d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:31.103165 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a97445f95204063b4c13c61d99271d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:31.105883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:31.114457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a97445f95204063b4c13c61d99271d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:31.115313 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a97445f95204063b4c13c61d99271d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:31.128894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ff2d76fc-a023-461c-9093-63dd90d423ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a97445f95204063b4c13c61d99271d5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:31.135489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1212/2430] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:30 2022] DELETE /identity/v3/users/3a97445f95204063b4c13c61d99271d5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:31.795061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:31.797750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:31.816050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac83d8-6587-4a0b-81c5-19fa0a7b6a17 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:31.891773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:31.892457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:31.892738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:31.893030 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] PATH_INFO: `/v3/projects/4aa613bc5a684d408d7e8b545c1c3a76` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:31.894182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa613bc5a684d408d7e8b545c1c3a76)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:31.896183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:31.999837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa613bc5a684d408d7e8b545c1c3a76', 'request_id': 'req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:32.006936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa613bc5a684d408d7e8b545c1c3a76', 'request_id': 'req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:32.013915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa613bc5a684d408d7e8b545c1c3a76', 'request_id': 'req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:32.076349 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] The token cache is being invalidate because project 4aa613bc5a684d408d7e8b545c1c3a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:32.077698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:32.078890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fcd3dfb5-abf9-4584-a3f3-2bc27a9d21f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:32.080426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1219/2431] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:31 2022] DELETE /identity/v3/projects/4aa613bc5a684d408d7e8b545c1c3a76 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:32.085490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.086003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.086133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.111623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:32.112819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:32.178982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1213/2432] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:32.188816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.189130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.189446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.214553 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:32.215935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:32.228595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1220/2433] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:32.233698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:32.244662 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c26b4-8116-46ef-8d81-44f99ce28bea None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:32.249857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:32.250462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.250742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.251021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.251644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:31:32.251992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:32.253550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:32.266528 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1214/2434] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:31:32.270602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:32.281186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71b245-2158-4a5c-80e6-9ca19b890a0a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:32.286941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:32.287449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.287732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.288005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.288647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:32.290607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:32.432563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1221/2435] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:32 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:32.436771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:32.447440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a7a3-cfc7-4d63-a6aa-3ad1e175f29d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:32.510457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:32.511238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.511636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.512006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.512762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:32.515312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:32.594944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1215/2436] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:32 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:32.596362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:32.607041 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e91d3-12ec-44d4-ae74-6c472ba2c6dc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:32.613360 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:32.618517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.618517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.618517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.618517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:32.618517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:32.628810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1222/2437] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:32.633762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:32.645380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dd55e-34be-4747-8914-5df4ded17497 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:32.655013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87124cfb-ea64-48f4-90f8-85c9354b0647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:32.655291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87124cfb-ea64-48f4-90f8-85c9354b0647 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.655479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87124cfb-ea64-48f4-90f8-85c9354b0647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.655755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87124cfb-ea64-48f4-90f8-85c9354b0647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.656417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87124cfb-ea64-48f4-90f8-85c9354b0647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:32.658414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87124cfb-ea64-48f4-90f8-85c9354b0647 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:32.670123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1216/2438] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:32.675074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:32.686783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a0b4-ab51-4174-9753-e97772f8c11d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:32.691776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:32.695273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.695273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.695273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] PATH_INFO: `/v3/projects/ef7316b700dc4a279fbea0cb7c427e50/users/5563ebb0a60c46cf9ebbb4c8cd8e9387/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.705370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:32.709595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7316b700dc4a279fbea0cb7c427e50, user_id=5563ebb0a60c46cf9ebbb4c8cd8e9387, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:32.712400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:32.786837 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1223/2439] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:32 2022] PUT /identity/v3/projects/ef7316b700dc4a279fbea0cb7c427e50/users/5563ebb0a60c46cf9ebbb4c8cd8e9387/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:32.798912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:32.799221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:32.799817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:32.829984 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] MFA Rules not processed for user `5563ebb0a60c46cf9ebbb4c8cd8e9387`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:32.831347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:32.863334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:32.902632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1217/2440] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:32.907999 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.018083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516a8c-bec5-46a0-bbe4-7efed1a8020e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.024589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.025578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.025864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.026133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.026870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:33.028782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:33.161109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1224/2441] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:32 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:33.164409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.178718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13646fa7-55c5-4bf0-b474-365cda8200b0 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.242619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.243244 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.243705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.243975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.244800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:33.246851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:33.348884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1218/2442] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:33 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:31:33.350198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.361687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7066728-4c2e-45b5-aa86-d7badd89cd53 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.367873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.368717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.369213 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.369658 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.370476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:33.372366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:33.386128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1225/2443] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:33.394037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.408453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc2e17-ccc7-42c4-b535-7c82c4f4a489 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.412394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f7cbe-bd44-4731-a126-bc01cb3abf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.413048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f7cbe-bd44-4731-a126-bc01cb3abf4b admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.413362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f7cbe-bd44-4731-a126-bc01cb3abf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.413615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f7cbe-bd44-4731-a126-bc01cb3abf4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.414272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f7cbe-bd44-4731-a126-bc01cb3abf4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:33.415892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f7cbe-bd44-4731-a126-bc01cb3abf4b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:33.427646 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1219/2444] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:33.434665 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.448575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25615f-d796-481d-ae4e-2710691e2ac7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.453969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.460647 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.460944 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.461893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] PATH_INFO: `/v3/projects/eb02056190c24090be2f8dcc8b93596a/users/8f765cb7ed214c4cbcc89275d93d17ce/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.480498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb02056190c24090be2f8dcc8b93596a, user_id=8f765cb7ed214c4cbcc89275d93d17ce, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:33.485631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:33.576167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1226/2445] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:33 2022] PUT /identity/v3/projects/eb02056190c24090be2f8dcc8b93596a/users/8f765cb7ed214c4cbcc89275d93d17ce/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:31:33.586853 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.587182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.587461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.618405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] MFA Rules not processed for user `8f765cb7ed214c4cbcc89275d93d17ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:33.619789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:33.702401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1220/2446] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:33.706951 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.789319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831e869-2aa2-45ce-8a98-4d11dfa41ffb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.794310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.794936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.795364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.795653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.796791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:33.799515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:33.905083 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1227/2447] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:33 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:33.906279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:33.918994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f9025-5cde-4cce-bc9a-067511015546 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:33.970178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:33.995812 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:33.996512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.996874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.997355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.998085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.000576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.094955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1221/2448] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:33 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:34.098919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:34.109679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025f7d1-4202-414a-bbd3-383e8a05e93e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:34.114210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:34.114808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.115090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.115362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.116000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.117626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.128470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1228/2449] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:34.133153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:34.144059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb56ac6-03c0-4e45-af41-4164bf88b895 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:34.148789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:34.149486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.149818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.150091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] PATH_INFO: `/v3/projects/aeb52468bbe14394858114b4b0a3ef2b/users/cfb4f3e951d14152a8232ba57f8ea91c/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.166242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb52468bbe14394858114b4b0a3ef2b, user_id=cfb4f3e951d14152a8232ba57f8ea91c, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.168915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.236582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1222/2450] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:34 2022] PUT /identity/v3/projects/aeb52468bbe14394858114b4b0a3ef2b/users/cfb4f3e951d14152a8232ba57f8ea91c/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:31:34.240506 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:34.251350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f5f9-6c7b-4eb4-a7ef-cbfebe3b70dc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:34.265554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:34.308005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89a63c-636c-4d48-8c8d-1d52c404585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:34.308611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89a63c-636c-4d48-8c8d-1d52c404585a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.308888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89a63c-636c-4d48-8c8d-1d52c404585a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.309192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89a63c-636c-4d48-8c8d-1d52c404585a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.309831 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89a63c-636c-4d48-8c8d-1d52c404585a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.313494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89a63c-636c-4d48-8c8d-1d52c404585a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.322997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1229/2451] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:34.327755 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:34.338290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8612f3-f5ed-4aff-8d27-a79a4a406387 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:34.343290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:34.343947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.344228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.344500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] PATH_INFO: `/v3/projects/aeb52468bbe14394858114b4b0a3ef2b/users/cfb4f3e951d14152a8232ba57f8ea91c/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.346416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb52468bbe14394858114b4b0a3ef2b, user_id=cfb4f3e951d14152a8232ba57f8ea91c, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.349093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.359711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1223/2452] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:34 2022] PUT /identity/v3/projects/aeb52468bbe14394858114b4b0a3ef2b/users/cfb4f3e951d14152a8232ba57f8ea91c/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:34.429242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.429535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.429817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.458633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] MFA Rules not processed for user `cfb4f3e951d14152a8232ba57f8ea91c`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:34.459852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:34.531982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1230/2453] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:34.543273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:34.545022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:34.603146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96bc0-99f4-4ce0-a25f-0c801787173f admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:34.606876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:34.607479 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.608304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.609177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.619226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.620656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.631644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1224/2454] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:34.730933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:34.732429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:34.745274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dada53-c246-4aa2-af73-d7f983bab223 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:34.749614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b05e6-2bb1-48d0-b5d6-246bcdc56650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:34.750134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b05e6-2bb1-48d0-b5d6-246bcdc56650 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:34.750410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b05e6-2bb1-48d0-b5d6-246bcdc56650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:34.750683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b05e6-2bb1-48d0-b5d6-246bcdc56650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:34.806838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b05e6-2bb1-48d0-b5d6-246bcdc56650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:34.808173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b05e6-2bb1-48d0-b5d6-246bcdc56650 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:34.817750 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1231/2455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:38.609866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:38.677864 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dbaa7-7657-4958-96a7-6425f0b347af service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:38.682883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:38.683548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:38.683834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:38.684112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] PATH_INFO: `/v3/users/5563ebb0a60c46cf9ebbb4c8cd8e9387` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:38.685303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5563ebb0a60c46cf9ebbb4c8cd8e9387)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:38.687994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:38.847477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:38.856226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5563ebb0a60c46cf9ebbb4c8cd8e9387'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:38.951075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5563ebb0a60c46cf9ebbb4c8cd8e9387'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:38.958800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5563ebb0a60c46cf9ebbb4c8cd8e9387'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:38.959445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5563ebb0a60c46cf9ebbb4c8cd8e9387'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:38.971966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5563ebb0a60c46cf9ebbb4c8cd8e9387'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:38.977819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1225/2456] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:38 2022] DELETE /identity/v3/users/5563ebb0a60c46cf9ebbb4c8cd8e9387 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:38.982245 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:38.983334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:38.983558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.002962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:39.004145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:39.033460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:39.064033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1232/2457] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:39.073706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:39.075117 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:39.132159 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660bb85-8d73-4c41-aa1c-bd6e769f9f57 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:39.135730 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:39.136331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:39.136614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:39.136889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.146718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:39.148175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:39.158579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1226/2458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:39.569636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:39.571153 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:39.583920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163fff6-1fd1-4ad7-9a15-79b377cbce16 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:39.588534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:39.589198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:39.589505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:39.589778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] PATH_INFO: `/v3/projects/ef7316b700dc4a279fbea0cb7c427e50` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.590894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7316b700dc4a279fbea0cb7c427e50)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:39.593424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:39.700613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7316b700dc4a279fbea0cb7c427e50', 'request_id': 'req-fa6e640a-4420-4002-87f8-34a498ee1cba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:39.706231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7316b700dc4a279fbea0cb7c427e50', 'request_id': 'req-fa6e640a-4420-4002-87f8-34a498ee1cba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:39.712667 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7316b700dc4a279fbea0cb7c427e50', 'request_id': 'req-fa6e640a-4420-4002-87f8-34a498ee1cba'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:39.772715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] The token cache is being invalidate because project ef7316b700dc4a279fbea0cb7c427e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:39.773036 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:39.773650 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:39.775155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1233/2459] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:39 2022] DELETE /identity/v3/projects/ef7316b700dc4a279fbea0cb7c427e50 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:39.782907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:39.792830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30eb76a-e362-43e5-bc86-42a6c6a07cbf service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:39.850996 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:39.851636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:39.851978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:39.852252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] PATH_INFO: `/v3/users/8f765cb7ed214c4cbcc89275d93d17ce` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.853391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f765cb7ed214c4cbcc89275d93d17ce)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:39.855373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:39.998956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f765cb7ed214c4cbcc89275d93d17ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.104858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f765cb7ed214c4cbcc89275d93d17ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.107254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:40.112909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f765cb7ed214c4cbcc89275d93d17ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.113587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f765cb7ed214c4cbcc89275d93d17ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.126133 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f765cb7ed214c4cbcc89275d93d17ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.131961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1227/2460] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:39 2022] DELETE /identity/v3/users/8f765cb7ed214c4cbcc89275d93d17ce => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:40.558957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:40.560455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:40.575900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e640a-4420-4002-87f8-34a498ee1cba admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:40.642847 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:40.643820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:40.644110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:40.644389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] PATH_INFO: `/v3/projects/eb02056190c24090be2f8dcc8b93596a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:40.645617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb02056190c24090be2f8dcc8b93596a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:40.648045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:40.756709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb02056190c24090be2f8dcc8b93596a', 'request_id': 'req-9f665f3a-91fd-4d21-b84b-a60625d8a79a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.766772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb02056190c24090be2f8dcc8b93596a', 'request_id': 'req-9f665f3a-91fd-4d21-b84b-a60625d8a79a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.768502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:40.774882 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb02056190c24090be2f8dcc8b93596a', 'request_id': 'req-9f665f3a-91fd-4d21-b84b-a60625d8a79a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.894810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] The token cache is being invalidate because project eb02056190c24090be2f8dcc8b93596a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:40.895159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.895800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:40.897435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1234/2461] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:40 2022] DELETE /identity/v3/projects/eb02056190c24090be2f8dcc8b93596a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:40.901454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:40.902953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:40.917616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f17884-dfb8-4db8-b36d-ca004cd17af5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:40.979707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:40.980345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:40.980628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:40.980913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] PATH_INFO: `/v3/users/cfb4f3e951d14152a8232ba57f8ea91c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:40.982170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb4f3e951d14152a8232ba57f8ea91c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:40.984689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:41.154839 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb4f3e951d14152a8232ba57f8ea91c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.223783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb4f3e951d14152a8232ba57f8ea91c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.234430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb4f3e951d14152a8232ba57f8ea91c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.235354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb4f3e951d14152a8232ba57f8ea91c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.249247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa8bad30-6b44-4d76-badb-17d127030a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb4f3e951d14152a8232ba57f8ea91c'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.255316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1228/2462] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:40 2022] DELETE /identity/v3/users/cfb4f3e951d14152a8232ba57f8ea91c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:41.667012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:41.668457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:41.682062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f665f3a-91fd-4d21-b84b-a60625d8a79a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:41.738901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:41.739417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:41.739694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:41.739964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] PATH_INFO: `/v3/projects/aeb52468bbe14394858114b4b0a3ef2b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:41.741095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb52468bbe14394858114b4b0a3ef2b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:41.742980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:41.842401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb52468bbe14394858114b4b0a3ef2b', 'request_id': 'req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.848069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb52468bbe14394858114b4b0a3ef2b', 'request_id': 'req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.854330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb52468bbe14394858114b4b0a3ef2b', 'request_id': 'req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.916075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] The token cache is being invalidate because project aeb52468bbe14394858114b4b0a3ef2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:41.916445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.917258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ac18e84-4dee-4e78-8b41-05684ea0cf7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:41.918798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1235/2463] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:41 2022] DELETE /identity/v3/projects/aeb52468bbe14394858114b4b0a3ef2b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:41.929642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:41.929642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:41.929642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:41.947334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:41.948565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.002134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:42.016201 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1229/2464] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:42.024347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.024637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.024983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.046390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:42.047605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.063128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1236/2465] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:42.064687 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.074513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aeb86-14f8-4705-9ba9-3d1b91a680cb None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.080021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.080619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.080898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.081202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.081855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:31:42.082158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.083754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.095721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1230/2466] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:31:42.100263 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.110609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed46f-5121-4366-b8ab-d71f41053662 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.115269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.115852 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.116156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.116428 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.117088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.119054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.253197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1237/2467] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:42 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:42.256659 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.266995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729c7f8-49b1-4992-864d-38c1d7088442 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.346932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.347504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.347789 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.357043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.357043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.357043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.431827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1231/2468] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:42 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:42.437119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.448546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c21e06-ee13-4e3b-b658-601f5d87ac35 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.454185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.454790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.455134 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.455447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.456095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.457973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.469162 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1238/2469] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:42.473128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.490261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a706-3fa5-4664-8871-5ff387a407f8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.490261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8e404-070f-431a-85b4-4ca16ba78151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.490261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8e404-070f-431a-85b4-4ca16ba78151 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.490261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8e404-070f-431a-85b4-4ca16ba78151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.490703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8e404-070f-431a-85b4-4ca16ba78151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.490703 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8e404-070f-431a-85b4-4ca16ba78151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.492100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8e404-070f-431a-85b4-4ca16ba78151 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.503769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1232/2470] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:42.507851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.520651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60436-6855-4b5d-b147-f59e1ca577d0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.525351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.526073 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.526361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.526632 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] PATH_INFO: `/v3/projects/48903e02ec90431a96e8ff4776bb8288/users/dd6232829d314256b03d9e8123003148/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.542331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48903e02ec90431a96e8ff4776bb8288, user_id=dd6232829d314256b03d9e8123003148, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.545194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.619936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1239/2471] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:42 2022] PUT /identity/v3/projects/48903e02ec90431a96e8ff4776bb8288/users/dd6232829d314256b03d9e8123003148/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:42.632285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.632661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.632975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.665931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] MFA Rules not processed for user `dd6232829d314256b03d9e8123003148`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:42.667188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.733567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1233/2472] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:42.745270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.746927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.762681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:42.806890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40b51a-9c90-4862-9fbd-505e6971285a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.810694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462de54-f30c-4ed6-9a58-de294ffbf5f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.811298 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462de54-f30c-4ed6-9a58-de294ffbf5f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.811581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462de54-f30c-4ed6-9a58-de294ffbf5f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.811903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462de54-f30c-4ed6-9a58-de294ffbf5f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.820155 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462de54-f30c-4ed6-9a58-de294ffbf5f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.821635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462de54-f30c-4ed6-9a58-de294ffbf5f5 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:42.831273 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1240/2473] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:42.855981 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.905915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:42.920521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c5a2d-46f5-4fe7-bc68-c42ee9380a00 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.925250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.925907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.926182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.926454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] PATH_INFO: `/v3/users/dd6232829d314256b03d9e8123003148` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.927579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6232829d314256b03d9e8123003148)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.929534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:43.113501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6232829d314256b03d9e8123003148'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.167308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6232829d314256b03d9e8123003148'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.175554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6232829d314256b03d9e8123003148'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.176228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6232829d314256b03d9e8123003148'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.188949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6232829d314256b03d9e8123003148'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.194807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1234/2474] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:42 2022] DELETE /identity/v3/users/dd6232829d314256b03d9e8123003148 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:43.199260 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:43.199547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:43.199817 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:43.221044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:43.222234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:43.282131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1241/2475] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:43.292123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:43.293536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:43.353314 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2ae54-b201-4993-abde-445697aedac7 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:43.357527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01afd6-8fa2-4c22-b4ea-fa67cc910428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:43.358135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01afd6-8fa2-4c22-b4ea-fa67cc910428 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:43.358415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01afd6-8fa2-4c22-b4ea-fa67cc910428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:43.359241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01afd6-8fa2-4c22-b4ea-fa67cc910428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:43.370595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01afd6-8fa2-4c22-b4ea-fa67cc910428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:43.372006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01afd6-8fa2-4c22-b4ea-fa67cc910428 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:43.383217 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1235/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:43.815578 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:43.817198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:43.832720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac9967-b8bc-4458-afc3-ccad93b15a5c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:43.846772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:43.847627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:43.848029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:43.848474 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] PATH_INFO: `/v3/projects/48903e02ec90431a96e8ff4776bb8288` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:43.849877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48903e02ec90431a96e8ff4776bb8288)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:43.851826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:43.944712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48903e02ec90431a96e8ff4776bb8288', 'request_id': 'req-b61774f5-d2db-4643-b5be-159d36e90862'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.950241 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48903e02ec90431a96e8ff4776bb8288', 'request_id': 'req-b61774f5-d2db-4643-b5be-159d36e90862'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:43.957074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48903e02ec90431a96e8ff4776bb8288', 'request_id': 'req-b61774f5-d2db-4643-b5be-159d36e90862'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:44.012711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] The token cache is being invalidate because project 48903e02ec90431a96e8ff4776bb8288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:44.013024 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:44.013649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b61774f5-d2db-4643-b5be-159d36e90862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:44.015862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1242/2477] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:43 2022] DELETE /identity/v3/projects/48903e02ec90431a96e8ff4776bb8288 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:31:44.020716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.021030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.023689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.041592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:44.042808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:44.103883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1236/2478] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:44.118446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.118734 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.119011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.138942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:44.140779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:44.148598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1243/2479] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:44.152742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.162610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdff75-bb9a-49c2-8706-c70c3af91cdb None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.167174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.167764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.168043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.168310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.168949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:31:44.169279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.170863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.177565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:44.182519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1237/2480] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:31:44.187733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.196965 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8506cdf-1e7f-4b8a-996a-47484d7b4e87 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.201542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.202130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.203122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.203446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.204068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.205994 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.301244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1244/2481] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:31:44 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:31:44.305650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.317279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e7549-2c1d-438e-a958-04d49de15b4a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.373636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.374912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.375384 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.375861 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.376483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.378417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.459274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1238/2482] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:31:44 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:31:44.463481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.474364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a5d54-2ea9-41b2-a236-2cb5a741142a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.479066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.479662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.479941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.480284 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.480972 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.482606 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.488393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:44.493355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1245/2483] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:44.497241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.508526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440b4e-e953-4161-8ce6-88f294550e0c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.513285 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4c1f9-4f96-44b9-a49c-ebbda42e9987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.513874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4c1f9-4f96-44b9-a49c-ebbda42e9987 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.514153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4c1f9-4f96-44b9-a49c-ebbda42e9987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.514424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4c1f9-4f96-44b9-a49c-ebbda42e9987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.515436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4c1f9-4f96-44b9-a49c-ebbda42e9987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.517045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4c1f9-4f96-44b9-a49c-ebbda42e9987 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.527490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1239/2484] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:31:44.531830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.542559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8acc6f-49c7-45c8-abeb-0a700d7862b2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.547737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.548381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.548786 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.549078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] PATH_INFO: `/v3/projects/b72df548c9f74b06858740e09546179b/users/a37b3d41c18541828b0aff71b4419bb8/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.564093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72df548c9f74b06858740e09546179b, user_id=a37b3d41c18541828b0aff71b4419bb8, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.566862 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.643140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1246/2485] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:31:44 2022] PUT /identity/v3/projects/b72df548c9f74b06858740e09546179b/users/a37b3d41c18541828b0aff71b4419bb8/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:44.653792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e843e-be8d-43bc-981f-26de24877573 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.654199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e843e-be8d-43bc-981f-26de24877573 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.654719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e843e-be8d-43bc-981f-26de24877573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.683104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f89e843e-be8d-43bc-981f-26de24877573 None None] MFA Rules not processed for user `a37b3d41c18541828b0aff71b4419bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:44.684320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f89e843e-be8d-43bc-981f-26de24877573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:44.748873 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1240/2486] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:44.760043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.761768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:44.819106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6944d2-2e79-4533-abda-d3c9c1821ec8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.822865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25336736-0536-45fe-be86-b3c11e1c7c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.823467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25336736-0536-45fe-be86-b3c11e1c7c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.823742 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25336736-0536-45fe-be86-b3c11e1c7c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.824010 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25336736-0536-45fe-be86-b3c11e1c7c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.832392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25336736-0536-45fe-be86-b3c11e1c7c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.833855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25336736-0536-45fe-be86-b3c11e1c7c57 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.843414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1247/2487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:46.626595 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:46.627091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:46.627091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:46.670692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] MFA Rules not processed for user `a37b3d41c18541828b0aff71b4419bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:46.672787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:46.685949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1241/2488] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:48.600790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:48.601304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:48.601445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:48.620275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:48.621544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:48.677449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1248/2489] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 10 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:48.851947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:48.853418 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:48.919261 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:48.925835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff0f82-82d9-45c8-8cca-481e0d1dcc3b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:48.930832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:48.931610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:48.931740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:48.932113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:48.940977 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:48.942679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:48.951579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1242/2490] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:31:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:31:58.158858 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:58.160801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:58.271765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:58.289605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:58.299438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2832991-3f13-474d-bb63-4e2a05219df2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:58.307257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:58.310932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:58.317459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:58.318326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:58.320638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:58.352046 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7dbcb-a507-46d3-8532-b5d1127b42d7 service swift] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:58.357926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:58.358611 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:58.358986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:58.359291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:58.359792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:58.361128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:58.361947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:58.373573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1249/2491] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:58.375521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:58.377367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:58.393580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:58.452170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:58.454374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:58.462871 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64fd55-9ea8-413d-98f8-4fcbeefccb29 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:58.467521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:58.468203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:58.468589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:58.468958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] PATH_INFO: `/v3/users/379f81ea933d4207865a8d1f299b5634` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:58.470147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=379f81ea933d4207865a8d1f299b5634)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:58.472101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:58.475571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1243/2492] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:58.676489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379f81ea933d4207865a8d1f299b5634'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:58.684164 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379f81ea933d4207865a8d1f299b5634'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:58.774609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379f81ea933d4207865a8d1f299b5634'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:58.775311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379f81ea933d4207865a8d1f299b5634'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:58.789363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379f81ea933d4207865a8d1f299b5634'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:58.796418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1250/2493] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:58 2022] DELETE /identity/v3/users/379f81ea933d4207865a8d1f299b5634 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:59.203855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:59.223355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7bca-4ca4-430f-91f9-676f767d10c3 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:59.284387 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:59.285149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:59.285496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:59.285990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] PATH_INFO: `/v3/projects/6ccc46e1ccb04fb386367a71ad13f008` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:59.287323 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ccc46e1ccb04fb386367a71ad13f008)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:59.289395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:59.409216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:59.415429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ccc46e1ccb04fb386367a71ad13f008', 'request_id': 'req-de3895fb-8834-40a9-ac13-a623c881d088'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.420953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ccc46e1ccb04fb386367a71ad13f008', 'request_id': 'req-de3895fb-8834-40a9-ac13-a623c881d088'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.428587 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ccc46e1ccb04fb386367a71ad13f008', 'request_id': 'req-de3895fb-8834-40a9-ac13-a623c881d088'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.535161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] The token cache is being invalidate because project 6ccc46e1ccb04fb386367a71ad13f008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:31:59.535574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.536429 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.538239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1244/2494] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:31:59 2022] DELETE /identity/v3/projects/6ccc46e1ccb04fb386367a71ad13f008 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:59.542989 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:59.545097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:59.561186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f95a0e-2b86-4119-b740-8cca90d2d5cf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:59.652671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:59.655637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:59.656146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:59.656430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] PATH_INFO: `/v3/users/3dc49964b34444a492ea183dc933b9e1` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:59.657886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc49964b34444a492ea183dc933b9e1)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:59.660270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:59.906669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:59.919278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc49964b34444a492ea183dc933b9e1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.927328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc49964b34444a492ea183dc933b9e1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.990772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc49964b34444a492ea183dc933b9e1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:31:59.991405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc49964b34444a492ea183dc933b9e1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.015022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc49964b34444a492ea183dc933b9e1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.023863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1251/2495] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:31:59 2022] DELETE /identity/v3/users/3dc49964b34444a492ea183dc933b9e1 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:00.324383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:00.335203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:00.344406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3895fb-8834-40a9-ac13-a623c881d088 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:00.424257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:00.424913 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:00.425223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:00.425499 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] PATH_INFO: `/v3/projects/9baf1533c0ec48b3b1ac29b251a9e5b9` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:00.426649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9baf1533c0ec48b3b1ac29b251a9e5b9)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:00.428596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:00.557711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9baf1533c0ec48b3b1ac29b251a9e5b9', 'request_id': 'req-91d6aaa1-5644-4024-82ca-5186ea15497a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.567255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9baf1533c0ec48b3b1ac29b251a9e5b9', 'request_id': 'req-91d6aaa1-5644-4024-82ca-5186ea15497a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.575086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9baf1533c0ec48b3b1ac29b251a9e5b9', 'request_id': 'req-91d6aaa1-5644-4024-82ca-5186ea15497a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.667324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] The token cache is being invalidate because project 9baf1533c0ec48b3b1ac29b251a9e5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:00.667629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.668412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:00.670458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1245/2496] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:00 2022] DELETE /identity/v3/projects/9baf1533c0ec48b3b1ac29b251a9e5b9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:02.440304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:02.441866 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:02.456986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358d32-d707-42dd-a24c-8c2e30c244cb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:02.514806 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:02.516027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:02.516329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:02.516608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] PATH_INFO: `/v3/users/0ace2374550e40ce812407406c262ac5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:02.517848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ace2374550e40ce812407406c262ac5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:02.519756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:02.741955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ace2374550e40ce812407406c262ac5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:02.748839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ace2374550e40ce812407406c262ac5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:02.818707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ace2374550e40ce812407406c262ac5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:02.819326 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ace2374550e40ce812407406c262ac5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:02.831433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ec7f51f-79ec-49b2-a79d-bbcd688da02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ace2374550e40ce812407406c262ac5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:02.837267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1252/2497] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:02 2022] DELETE /identity/v3/users/0ace2374550e40ce812407406c262ac5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:03.060407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.062057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:03.075286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6aaa1-5644-4024-82ca-5186ea15497a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.134484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.135113 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.135393 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.135665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] PATH_INFO: `/v3/projects/f3070abbdec34c779d4bd382b2bb1697` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.136766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3070abbdec34c779d4bd382b2bb1697)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.138712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.140090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:03.224889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3070abbdec34c779d4bd382b2bb1697', 'request_id': 'req-b31eecae-8517-41c1-977c-14e04d033de4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:03.230164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3070abbdec34c779d4bd382b2bb1697', 'request_id': 'req-b31eecae-8517-41c1-977c-14e04d033de4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:03.236326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3070abbdec34c779d4bd382b2bb1697', 'request_id': 'req-b31eecae-8517-41c1-977c-14e04d033de4'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:03.302826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] The token cache is being invalidate because project f3070abbdec34c779d4bd382b2bb1697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:03.303154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:03.303776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-b31eecae-8517-41c1-977c-14e04d033de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:03.305683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1246/2498] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:03 2022] DELETE /identity/v3/projects/f3070abbdec34c779d4bd382b2bb1697 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:03.310771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.311056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.311327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.335304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:03.336551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:03.362158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:03.395278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1253/2499] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:03.403279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.403570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.403844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.424174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:03.427131 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:03.433437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1247/2500] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:03.441157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.451389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d229753-08d1-4c94-93b8-01c216cc533b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.456424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.457092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.457379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.457654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.458308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:32:03.458621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.460243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.472285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1254/2501] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:32:03.476704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.487255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7bdc8-25ea-4615-b18e-7e146b27415e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.492070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.492668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.492950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.493255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.493890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.495795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.590794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1248/2502] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:03 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:03.595877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.606268 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6415dd-9cab-4995-bb1e-0bc1ff415eb4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.674653 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.675269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.675555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.675840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.676464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.678402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.760655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1255/2503] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:03 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:03.764591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.775902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a739-640b-43ea-b61c-2149968a0dbe admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.780608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.781255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.781541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.781816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.782459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.784402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.794277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1249/2504] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:03.798344 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.808103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4cf0f-63c0-4523-ba59-a6554c51c0cc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.813401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d636ef-7451-4940-8768-c1e07a710764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.813992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d636ef-7451-4940-8768-c1e07a710764 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.814276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d636ef-7451-4940-8768-c1e07a710764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.814552 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d636ef-7451-4940-8768-c1e07a710764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.815194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d636ef-7451-4940-8768-c1e07a710764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.816777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d636ef-7451-4940-8768-c1e07a710764 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.831201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1256/2505] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:03.832378 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:03.842707 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d9386-eb9e-4adc-8ebf-fae94aeb4583 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:03.847598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:03.848251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.848544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.848825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] PATH_INFO: `/v3/projects/e3d92122eca2433dab79cfa4d9601159/users/5739e405875f414abdc70552a3c073eb/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.864311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d92122eca2433dab79cfa4d9601159, user_id=5739e405875f414abdc70552a3c073eb, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:03.867103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:03.956654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1250/2506] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:03 2022] PUT /identity/v3/projects/e3d92122eca2433dab79cfa4d9601159/users/5739e405875f414abdc70552a3c073eb/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:32:03.967772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:03.968228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:03.968507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:03.997513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] MFA Rules not processed for user `5739e405875f414abdc70552a3c073eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:03.998748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.067348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1257/2507] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:04.078800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:04.080493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.154379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0561633-1a1b-4af6-9610-f3da6260a3c3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:04.160873 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767195a1-2c99-4695-80e6-d505ca857cfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:04.169203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767195a1-2c99-4695-80e6-d505ca857cfa service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.169203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767195a1-2c99-4695-80e6-d505ca857cfa service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.169203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767195a1-2c99-4695-80e6-d505ca857cfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.170271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767195a1-2c99-4695-80e6-d505ca857cfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:04.172417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767195a1-2c99-4695-80e6-d505ca857cfa service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:04.183022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1251/2508] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:04.431207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:04.466872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:04.508912 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3957ed-727e-47eb-9212-f6bccb401ceb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:04.513699 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:04.514425 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.514985 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.516305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] PATH_INFO: `/v3/users/5739e405875f414abdc70552a3c073eb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.517991 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5739e405875f414abdc70552a3c073eb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:04.519993 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:04.685875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5739e405875f414abdc70552a3c073eb'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:04.694061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5739e405875f414abdc70552a3c073eb'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:04.751047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5739e405875f414abdc70552a3c073eb'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:04.751735 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5739e405875f414abdc70552a3c073eb'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:04.765444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5739e405875f414abdc70552a3c073eb'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:04.771948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1258/2509] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:04 2022] DELETE /identity/v3/users/5739e405875f414abdc70552a3c073eb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:04.776455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.776747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.777048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.798121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:04.799325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.804440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:04.857874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1252/2510] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:04.868306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:04.869717 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.931302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ba37a-4414-4daa-9c2e-c0683a6449b9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:04.935308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5d213-7381-443e-af29-4367a8b189b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:04.935933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5d213-7381-443e-af29-4367a8b189b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.936214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5d213-7381-443e-af29-4367a8b189b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.936544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5d213-7381-443e-af29-4367a8b189b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.946180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5d213-7381-443e-af29-4367a8b189b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:04.947598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5d213-7381-443e-af29-4367a8b189b9 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:04.957053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1259/2511] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:05.396590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:05.398445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:05.412376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2d7c1-c5f4-44f3-85d4-22b201928d31 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:05.418498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:05.419223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:05.419626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:05.419948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] PATH_INFO: `/v3/projects/e3d92122eca2433dab79cfa4d9601159` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:05.421203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d92122eca2433dab79cfa4d9601159)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:05.423803 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:05.569485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d92122eca2433dab79cfa4d9601159', 'request_id': 'req-97d6ed4e-be16-474f-af3b-d8062e2425ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:05.574964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d92122eca2433dab79cfa4d9601159', 'request_id': 'req-97d6ed4e-be16-474f-af3b-d8062e2425ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:05.581439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d92122eca2433dab79cfa4d9601159', 'request_id': 'req-97d6ed4e-be16-474f-af3b-d8062e2425ce'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:05.698490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] The token cache is being invalidate because project e3d92122eca2433dab79cfa4d9601159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:05.698738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:05.699368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-97d6ed4e-be16-474f-af3b-d8062e2425ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:05.700893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1253/2512] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:05 2022] DELETE /identity/v3/projects/e3d92122eca2433dab79cfa4d9601159 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:05.705823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25356-e480-4796-9065-b4257675f37f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:05.706294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25356-e480-4796-9065-b4257675f37f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:05.706391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25356-e480-4796-9065-b4257675f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:05.726458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d7f25356-e480-4796-9065-b4257675f37f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:05.727675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d7f25356-e480-4796-9065-b4257675f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:05.773698 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:05.794531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1260/2513] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:05.798602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:05.799003 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:05.799386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:05.821619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:05.823269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:05.834738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1254/2514] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:05.838223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25356-e480-4796-9065-b4257675f37f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:05.849361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25356-e480-4796-9065-b4257675f37f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:05.853927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:05.854528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:05.854801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:05.855070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:05.855713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:32:05.856012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:05.857730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:05.872675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1261/2515] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:32:05.873950 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:05.885550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e50e4b-f4fe-45d7-8e81-950286636f56 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:05.891047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bfbbd-2e6f-4e28-893b-9029e39a6220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:05.891658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bfbbd-2e6f-4e28-893b-9029e39a6220 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:05.891963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bfbbd-2e6f-4e28-893b-9029e39a6220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:05.892259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bfbbd-2e6f-4e28-893b-9029e39a6220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:05.893059 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bfbbd-2e6f-4e28-893b-9029e39a6220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:05.894963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bfbbd-2e6f-4e28-893b-9029e39a6220 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.016772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1255/2516] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:05 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:32:06.021181 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.031546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a88f8-f1f6-4b7d-b87e-5b2a2b66e929 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.053633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.055050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.055407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.076471 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:06.078368 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:06.093711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.094302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.094600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.094856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.095489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.097536 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.134754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1256/2517] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:06.146759 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.148061 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:06.160964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:06.216305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfae22-7da8-46c2-ae6d-ed4549511bfc None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.221581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.222193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.222475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.222745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.223725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1262/2518] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:06 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:06.228090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.231847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.233485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.239446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf26e9-5a70-462a-83da-9719e560d8d8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.244592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1257/2519] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:32:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:32:06.246652 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.247253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.247533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.247805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.248537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.251388 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.262920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1263/2520] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:32:06.267214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.277831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ddcce-23cd-44c5-b97b-072a0e6fa3c1 service swift] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.282847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10df49-8917-45cb-aaf0-a972addc1994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.283469 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10df49-8917-45cb-aaf0-a972addc1994 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.283753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10df49-8917-45cb-aaf0-a972addc1994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.284296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10df49-8917-45cb-aaf0-a972addc1994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.284938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10df49-8917-45cb-aaf0-a972addc1994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.286565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10df49-8917-45cb-aaf0-a972addc1994 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.297540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1258/2521] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:06.301767 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.311657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4058c4-04f9-444d-8f5f-222543bc817c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.317484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.318019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.318367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.318646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] PATH_INFO: `/v3/projects/b8d4047a7ffa4a95ae0b4cbf3c39ad4c/users/3747aa23d07e4c7290472edb6ef58e8d/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.334450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d4047a7ffa4a95ae0b4cbf3c39ad4c, user_id=3747aa23d07e4c7290472edb6ef58e8d, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.337212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.413447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1264/2522] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:06 2022] PUT /identity/v3/projects/b8d4047a7ffa4a95ae0b4cbf3c39ad4c/users/3747aa23d07e4c7290472edb6ef58e8d/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:06.424513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.424920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.424920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.454041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] MFA Rules not processed for user `3747aa23d07e4c7290472edb6ef58e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:06.455257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:06.519652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1259/2523] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:06.524146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.591168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1776ce9-61d8-413c-af4c-879add0d965d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.596514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.597152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.597473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.597758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.598389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.600304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.715666 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1265/2524] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:06 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:06.716740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.730837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80307f0a-1c9c-45a1-8aa8-2ea9135f074e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.813332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.813945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.814228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.814513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.815132 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.817078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.927845 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1260/2525] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:06 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:32:06.931813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.944619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284c6d-1a2a-4d83-9906-f3e940d8cf03 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.947294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.947946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.948233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.948522 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.949186 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:06.950854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:06.962363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1266/2526] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:06.966615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:06.977538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3447a-8e45-4ef0-b655-a3aaae08b6b2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:06.982835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9347-7ec4-4e81-a75e-9858c516c98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:06.983337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9347-7ec4-4e81-a75e-9858c516c98f admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:06.983618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9347-7ec4-4e81-a75e-9858c516c98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:06.983928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9347-7ec4-4e81-a75e-9858c516c98f admin admin] PATH_INFO: `/v3/projects/dd52a0d027604a40a29dedb1923ed040/users/64ba3341f0f1497c88e391a43217b781/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:06.987271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:06.999277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9347-7ec4-4e81-a75e-9858c516c98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd52a0d027604a40a29dedb1923ed040, user_id=64ba3341f0f1497c88e391a43217b781, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.002110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9347-7ec4-4e81-a75e-9858c516c98f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.089477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1261/2527] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:06 2022] PUT /identity/v3/projects/dd52a0d027604a40a29dedb1923ed040/users/64ba3341f0f1497c88e391a43217b781/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:07.094323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:07.103515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:07.107232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcfc36-d413-4923-9e43-2bb967d177df admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:07.115074 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.115375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.115648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.136847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:07.138258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:07.173276 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:07.173890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.177893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.177893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.177893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.177893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.186747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1267/2528] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:07.191229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:07.198827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1262/2529] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 10 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:07.202327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ea434-c772-4a8e-81ee-2fa4c4e8f7d6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:07.207526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e67b2-1385-41d6-8f63-d7903fe4c16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:07.208189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e67b2-1385-41d6-8f63-d7903fe4c16e admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.212333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e67b2-1385-41d6-8f63-d7903fe4c16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.212333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e67b2-1385-41d6-8f63-d7903fe4c16e admin admin] PATH_INFO: `/v3/projects/dd52a0d027604a40a29dedb1923ed040/users/64ba3341f0f1497c88e391a43217b781/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.212333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e67b2-1385-41d6-8f63-d7903fe4c16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd52a0d027604a40a29dedb1923ed040, user_id=64ba3341f0f1497c88e391a43217b781, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.212877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:07.213954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e67b2-1385-41d6-8f63-d7903fe4c16e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.225681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1268/2530] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:07 2022] PUT /identity/v3/projects/dd52a0d027604a40a29dedb1923ed040/users/64ba3341f0f1497c88e391a43217b781/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:32:07.237367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.241055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.241055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.269449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] MFA Rules not processed for user `64ba3341f0f1497c88e391a43217b781`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:07.270832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:07.285422 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8085f-a3f4-4e82-a3d4-b4c6dac174d3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:07.347327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1269/2531] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:07.350720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:07.351449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.351804 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.352171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.358602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:07.360126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:07.386264 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:07.412259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.413964 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.420129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecc275-f027-4072-8327-281f501fd6b8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:07.422106 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1263/2532] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:32:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:32:07.427175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccea29-d8f8-400b-bd61-eb24c16b522c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:07.427789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccea29-d8f8-400b-bd61-eb24c16b522c service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.428070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccea29-d8f8-400b-bd61-eb24c16b522c service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.428341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccea29-d8f8-400b-bd61-eb24c16b522c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.438900 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccea29-d8f8-400b-bd61-eb24c16b522c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.440471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccea29-d8f8-400b-bd61-eb24c16b522c service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.451159 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1270/2533] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:32:07.695627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:07.706128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfe3e5-8cc6-4e60-a1a9-0c442c21ce6c service swift] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:07.710158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:07.710764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.711040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.711334 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.770439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.771921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.781991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1264/2534] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:08.122249 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:08.127251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:08.127251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:08.145020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:08.146312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:08.153517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1271/2535] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:32:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:08.165791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:08.177879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416427-2b74-4b9a-8ca2-82539b982791 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:08.184607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4788bf-d12b-44bb-8bc3-ef0f3eadcf3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:08.184607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4788bf-d12b-44bb-8bc3-ef0f3eadcf3f service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:08.184607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4788bf-d12b-44bb-8bc3-ef0f3eadcf3f service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:08.184607 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4788bf-d12b-44bb-8bc3-ef0f3eadcf3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:08.193878 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4788bf-d12b-44bb-8bc3-ef0f3eadcf3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:08.195510 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4788bf-d12b-44bb-8bc3-ef0f3eadcf3f service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:08.205354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1265/2536] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:32:08.227189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:08.229145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:08.242246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:08.307455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a52421-0f97-4880-a05d-cacd4d2a840e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:08.314106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:08.314722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:08.315006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:08.315275 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:08.316984 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:08.379669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:08.381228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:08.390538 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1272/2537] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:08.910929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:08.911412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:08.911533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:08.930529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:08.931737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:08.952377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1266/2538] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 10 13:32:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:08.953565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:08.966026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8640f77-43d1-4268-b2f1-d31c95d7082d service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:08.971261 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b7268-f234-46a7-b23b-f69627fc6dd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:08.971784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b7268-f234-46a7-b23b-f69627fc6dd8 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:08.972070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b7268-f234-46a7-b23b-f69627fc6dd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:08.972348 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b7268-f234-46a7-b23b-f69627fc6dd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:08.981583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b7268-f234-46a7-b23b-f69627fc6dd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:08.983156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b7268-f234-46a7-b23b-f69627fc6dd8 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:08.991889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1273/2539] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:32:11.735006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:11.736434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:11.800727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe665-f41e-4ccb-bd7d-3df9ebb333e9 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:11.805216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:11.805841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:11.806124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:11.806395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] PATH_INFO: `/v3/users/a37b3d41c18541828b0aff71b4419bb8` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:11.807520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37b3d41c18541828b0aff71b4419bb8)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:11.809459 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:11.974695 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37b3d41c18541828b0aff71b4419bb8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.047486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37b3d41c18541828b0aff71b4419bb8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.049112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:12.055822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37b3d41c18541828b0aff71b4419bb8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.056478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37b3d41c18541828b0aff71b4419bb8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.068886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37b3d41c18541828b0aff71b4419bb8'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.074848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1267/2540] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:11 2022] DELETE /identity/v3/users/a37b3d41c18541828b0aff71b4419bb8 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:12.080050 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:12.080537 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:12.080662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:12.099182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:12.100389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:12.159546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1274/2541] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:12.169289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:12.170681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:12.227817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf202c7-e1d7-4d50-a5f8-5b83de06ec20 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:12.231465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:12.232077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:12.232357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:12.232628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:12.241835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:12.243262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:12.252748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1268/2542] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:12.548076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:12.549555 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:12.561296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b6db-6530-428c-99e8-e2cff0e59291 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:12.566517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:12.567657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:12.567657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:12.567657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] PATH_INFO: `/v3/projects/b72df548c9f74b06858740e09546179b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:12.568870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72df548c9f74b06858740e09546179b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:12.570855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:12.661500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:12.676634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72df548c9f74b06858740e09546179b', 'request_id': 'req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.682395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72df548c9f74b06858740e09546179b', 'request_id': 'req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.689941 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72df548c9f74b06858740e09546179b', 'request_id': 'req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.766938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] The token cache is being invalidate because project b72df548c9f74b06858740e09546179b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:12.767237 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.767982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4a7de85d-11ac-4b4d-baa0-c98582ae5a56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:12.769762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1275/2543] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:12 2022] DELETE /identity/v3/projects/b72df548c9f74b06858740e09546179b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:22.248419 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=37,keystone:DELETE=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:22.789013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:28.237778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:28.254538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e69ca-f4d9-45f2-af72-3267efc29851 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:28.302815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:28.303456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:28.303746 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:28.303997 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:28.305765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:28.371892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:28.373416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:28.384143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1269/2544] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:29.299662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:29.300128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:29.300236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:29.319500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:29.320725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:29.381094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1276/2545] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:29.393537 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:29.394943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:29.473910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b50a5-19dc-4385-a6c6-6d6d184aa1b1 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:29.479926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:29.480641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:29.481034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:29.481373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:29.492534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:29.494218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:29.503698 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1270/2546] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:32:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:32:33.375521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:33.377161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:33.394890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e178a4-b07d-47c7-80a3-0e76771953f2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:33.400116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:33.400749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:33.401057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:33.401333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] PATH_INFO: `/v3/users/c9577bc4c4c546ca9fc220ebd3a16346` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:33.402465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9577bc4c4c546ca9fc220ebd3a16346)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:33.404438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:33.619762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9577bc4c4c546ca9fc220ebd3a16346'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:33.630460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9577bc4c4c546ca9fc220ebd3a16346'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:33.687269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9577bc4c4c546ca9fc220ebd3a16346'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:33.688142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9577bc4c4c546ca9fc220ebd3a16346'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:33.706376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-226d4b01-f072-4db6-bf8c-b1199ceedff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9577bc4c4c546ca9fc220ebd3a16346'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:33.735508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1277/2547] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:33 2022] DELETE /identity/v3/users/c9577bc4c4c546ca9fc220ebd3a16346 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:34.029573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.042881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e191bc-b208-4354-821b-36bb947e3116 service swift] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.102757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:34.152840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.153651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.154175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.154697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] PATH_INFO: `/v3/projects/26a97b47cc804e598c5e8680d279de19` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.156088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a97b47cc804e598c5e8680d279de19)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:34.158142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:34.293340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a97b47cc804e598c5e8680d279de19', 'request_id': 'req-ac196cfd-ac74-4d2c-a196-8654bfcf1080'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:34.299554 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a97b47cc804e598c5e8680d279de19', 'request_id': 'req-ac196cfd-ac74-4d2c-a196-8654bfcf1080'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:34.310937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a97b47cc804e598c5e8680d279de19', 'request_id': 'req-ac196cfd-ac74-4d2c-a196-8654bfcf1080'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:34.388740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] The token cache is being invalidate because project 26a97b47cc804e598c5e8680d279de19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:34.389116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:34.389776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac196cfd-ac74-4d2c-a196-8654bfcf1080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:34.391686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1271/2548] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:34 2022] DELETE /identity/v3/projects/26a97b47cc804e598c5e8680d279de19 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:34.396964 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.397490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.397590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.422105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:34.422105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:34.495142 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1278/2549] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:34.503017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.503305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.503582 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.528084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:34.529386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:34.543262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1272/2550] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:34.547288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.557656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c6136-ae63-4fc7-993c-41e773c52b9b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.562601 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.563447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.563961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.564417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.565315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:32:34.566011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:34.567575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:34.587278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1279/2551] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:32:34.591577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.601704 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007c8f0-f95d-4907-af44-0fa1c3574651 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.606893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.607579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.607866 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.608139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.608874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:34.610757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:34.695634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1273/2552] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:34 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:34.699708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.706892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:34.711640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96335cdb-43ca-4948-9d71-5f67e6e24c55 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.782115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.782790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.783067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.783340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.784234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:34.786830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:34.887438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1280/2553] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:34 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:34.891563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.909637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eea624-9750-49b3-9486-731cc4aae1cd admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.915444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.916242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.916719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.917024 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.917822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:34.919460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:34.935769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1274/2554] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:32:34.937101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.951898 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b2cc6-fe0c-4efa-9f39-b67f54575172 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.953921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46962d74-de92-4271-824f-bf1930bc7dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.954558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46962d74-de92-4271-824f-bf1930bc7dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.954853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46962d74-de92-4271-824f-bf1930bc7dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.955145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46962d74-de92-4271-824f-bf1930bc7dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:34.955954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46962d74-de92-4271-824f-bf1930bc7dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:34.957607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46962d74-de92-4271-824f-bf1930bc7dd3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:34.976064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1281/2555] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:34.977288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:34.988822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621ba63-f356-438e-89cf-33b2f700281d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:34.997197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:34.997197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:34.997197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:34.997197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] PATH_INFO: `/v3/projects/2b00137741fe4d079bf33bc0e4add53b/users/7080cd300ff8442d9eab1ec7a09ae257/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.014653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b00137741fe4d079bf33bc0e4add53b, user_id=7080cd300ff8442d9eab1ec7a09ae257, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:35.018071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:35.124926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1275/2556] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:34 2022] PUT /identity/v3/projects/2b00137741fe4d079bf33bc0e4add53b/users/7080cd300ff8442d9eab1ec7a09ae257/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:35.126927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.127313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.127688 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.168346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] MFA Rules not processed for user `7080cd300ff8442d9eab1ec7a09ae257`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:35.168346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:35.251107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1282/2557] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:35.263025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:35.264936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:35.347191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:35.352458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fab14-31c1-4bbf-8741-87912e2b3089 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:35.360277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98233c21-01e2-484e-a958-a6d9259b7ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:35.361077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233c21-01e2-484e-a958-a6d9259b7ece service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.361331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233c21-01e2-484e-a958-a6d9259b7ece service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.361632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233c21-01e2-484e-a958-a6d9259b7ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.371057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98233c21-01e2-484e-a958-a6d9259b7ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:35.372684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98233c21-01e2-484e-a958-a6d9259b7ece service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:35.384125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1276/2558] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:40.840729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:40.906005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf29b68-5d66-4b3c-be85-fb43ff3a831e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:40.910794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:40.912362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:40.912636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:40.913039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] PATH_INFO: `/v3/users/7080cd300ff8442d9eab1ec7a09ae257` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:40.914172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7080cd300ff8442d9eab1ec7a09ae257)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:40.916067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:40.930646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:41.146968 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7080cd300ff8442d9eab1ec7a09ae257'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:41.153946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7080cd300ff8442d9eab1ec7a09ae257'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:41.228496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7080cd300ff8442d9eab1ec7a09ae257'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:41.229177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7080cd300ff8442d9eab1ec7a09ae257'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:41.242574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7080cd300ff8442d9eab1ec7a09ae257'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:41.248571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1283/2559] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:40 2022] DELETE /identity/v3/users/7080cd300ff8442d9eab1ec7a09ae257 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:32:41.253031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:41.253487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:41.253606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:41.275401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:41.276609 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:41.338073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1277/2560] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:41.347758 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:41.349338 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:41.408673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894aa8b-2277-4218-82bc-4a0dc85a67c1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:41.413038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4107a59-5c9c-446c-ade1-458d3e7b407a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:41.414696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4107a59-5c9c-446c-ade1-458d3e7b407a service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:41.414696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4107a59-5c9c-446c-ade1-458d3e7b407a service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:41.414696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4107a59-5c9c-446c-ade1-458d3e7b407a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:41.423853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4107a59-5c9c-446c-ade1-458d3e7b407a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:41.425342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4107a59-5c9c-446c-ade1-458d3e7b407a service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:41.436014 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1284/2561] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:42.014294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.015959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:42.030883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4f306-eebc-42a1-95cd-f7ee358e4ba1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.037336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.037986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.038269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.038549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] PATH_INFO: `/v3/projects/2b00137741fe4d079bf33bc0e4add53b` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.039737 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b00137741fe4d079bf33bc0e4add53b)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.041722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.127932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b00137741fe4d079bf33bc0e4add53b', 'request_id': 'req-e6b9316a-77e8-452f-b55e-34d7923b2958'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:42.136840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b00137741fe4d079bf33bc0e4add53b', 'request_id': 'req-e6b9316a-77e8-452f-b55e-34d7923b2958'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:42.143078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b00137741fe4d079bf33bc0e4add53b', 'request_id': 'req-e6b9316a-77e8-452f-b55e-34d7923b2958'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:42.204488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] The token cache is being invalidate because project 2b00137741fe4d079bf33bc0e4add53b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:42.204810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:42.205463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-e6b9316a-77e8-452f-b55e-34d7923b2958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:42.207151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1278/2562] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:42 2022] DELETE /identity/v3/projects/2b00137741fe4d079bf33bc0e4add53b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:42.212619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.213111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.213227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.234255 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:42.235447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:42.295334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1285/2563] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:42.303880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.304399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.304508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.324215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:42.325508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:42.333533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1279/2564] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:42.337909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.347967 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152eed4b-7801-4f8a-a780-5edcf3cc57c4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.353370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.353971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.354254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.354528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.355180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:32:42.355488 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.357359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.371877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1286/2565] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:32:42.376097 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.385738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b7f42-de10-458e-92cd-99d1edf9d441 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.390575 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.391177 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.392238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.392238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.392492 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.394341 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.487828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1280/2566] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:42 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:42.493082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.504151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180d8ac-d779-4831-a6bc-38ba4f982917 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.523177 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:42.566755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.567361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.567641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.567929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.568745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.570691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.655635 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1287/2567] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:42 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:42.656933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.671514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a9959-0cd5-4f60-adfc-82df15704f62 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.678833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.678833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.679255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.679603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.680340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.681986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.692747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1281/2568] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:42.697412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.708192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b56e4-74f0-4c4f-9792-bd60a64d95cc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.713616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f139ea-9478-473f-9790-6eb2a991b1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.714395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f139ea-9478-473f-9790-6eb2a991b1fa admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.714787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f139ea-9478-473f-9790-6eb2a991b1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.715144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f139ea-9478-473f-9790-6eb2a991b1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.715892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f139ea-9478-473f-9790-6eb2a991b1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.717521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f139ea-9478-473f-9790-6eb2a991b1fa admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.732469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1288/2569] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:42.738258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.752048 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67394e2-2e35-4772-b22d-017be31c0911 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:42.759328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:42.760130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.762912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.762912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] PATH_INFO: `/v3/projects/b7c352f08c514569aa3833ea7beb52b5/users/920820f894244bd2a2a422b503c4b47a/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.777766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c352f08c514569aa3833ea7beb52b5, user_id=920820f894244bd2a2a422b503c4b47a, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:42.781218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:42.847416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1282/2570] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:42 2022] PUT /identity/v3/projects/b7c352f08c514569aa3833ea7beb52b5/users/920820f894244bd2a2a422b503c4b47a/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:32:42.857934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991fd24-c327-46f1-989e-4c41c7d86a42 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:42.858224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991fd24-c327-46f1-989e-4c41c7d86a42 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:42.858498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991fd24-c327-46f1-989e-4c41c7d86a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:42.888461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8991fd24-c327-46f1-989e-4c41c7d86a42 None None] MFA Rules not processed for user `920820f894244bd2a2a422b503c4b47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:42.889710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8991fd24-c327-46f1-989e-4c41c7d86a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:42.960043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1289/2571] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:42.971134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:42.976297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:42.987079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:43.031514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee8a71-61a0-473c-abd7-637998982101 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:43.035605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f923-4e9f-40ca-aa72-7730adaabf55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:43.036214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f923-4e9f-40ca-aa72-7730adaabf55 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:43.036498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f923-4e9f-40ca-aa72-7730adaabf55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:43.036814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f923-4e9f-40ca-aa72-7730adaabf55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:43.045337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f923-4e9f-40ca-aa72-7730adaabf55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:43.046814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f923-4e9f-40ca-aa72-7730adaabf55 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:43.055994 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1283/2572] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:47.592872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69835c06-0321-4626-aa9d-a5b30073182d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:47.593318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69835c06-0321-4626-aa9d-a5b30073182d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:47.593520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69835c06-0321-4626-aa9d-a5b30073182d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:47.594940 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1290/2573] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:47.602277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524004a-2172-4276-9431-477180b53695 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:47.602777 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524004a-2172-4276-9431-477180b53695 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:47.602876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524004a-2172-4276-9431-477180b53695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:47.624669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b524004a-2172-4276-9431-477180b53695 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:47.626107 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b524004a-2172-4276-9431-477180b53695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:47.720728 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1284/2574] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:47.734654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69835c06-0321-4626-aa9d-a5b30073182d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:47.736754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-69835c06-0321-4626-aa9d-a5b30073182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:47.804809 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69835c06-0321-4626-aa9d-a5b30073182d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:47.808544 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2245b-4f79-452c-932b-e9b611a3f500 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:47.809180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2245b-4f79-452c-932b-e9b611a3f500 service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:47.809457 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2245b-4f79-452c-932b-e9b611a3f500 service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:47.809726 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2245b-4f79-452c-932b-e9b611a3f500 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:47.820593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2245b-4f79-452c-932b-e9b611a3f500 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:47.822061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2245b-4f79-452c-932b-e9b611a3f500 service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:47.832873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1291/2575] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:48.082826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5744-5db9-4728-b867-f50fa16e9b03 None None] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.083121 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5744-5db9-4728-b867-f50fa16e9b03 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.083428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5744-5db9-4728-b867-f50fa16e9b03 None None] PATH_INFO: `/` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.084909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1285/2576] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:32:48.090077 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6354-1530-4108-b308-681b00c0136e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.090366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6354-1530-4108-b308-681b00c0136e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.090640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6354-1530-4108-b308-681b00c0136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.242192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a75b6354-1530-4108-b308-681b00c0136e None None] MFA Rules not processed for user `ffc4548fbe5b418ea9787d78e40838a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:48.243585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a75b6354-1530-4108-b308-681b00c0136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:48.275891 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:48.298938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1292/2577] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:48.310936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5744-5db9-4728-b867-f50fa16e9b03 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:48.312771 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c26c5744-5db9-4728-b867-f50fa16e9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:48.326514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5744-5db9-4728-b867-f50fa16e9b03 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:48.330025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfaed3-6410-4047-9148-88f499ba5b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:48.330620 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfaed3-6410-4047-9148-88f499ba5b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.330926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfaed3-6410-4047-9148-88f499ba5b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.331150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfaed3-6410-4047-9148-88f499ba5b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.339705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfaed3-6410-4047-9148-88f499ba5b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:48.341156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfaed3-6410-4047-9148-88f499ba5b50 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:48.350827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1286/2578] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:48.447038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d50710-d22a-420d-b17f-f3905bf42844 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.447393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d50710-d22a-420d-b17f-f3905bf42844 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.447701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d50710-d22a-420d-b17f-f3905bf42844 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.449129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1293/2579] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 10 13:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:48.454563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505822f9-e4ec-42ef-94e5-1035fc5c17d3 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.455044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505822f9-e4ec-42ef-94e5-1035fc5c17d3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.455318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505822f9-e4ec-42ef-94e5-1035fc5c17d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.477904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-505822f9-e4ec-42ef-94e5-1035fc5c17d3 None None] MFA Rules not processed for user `5dda95d65ff54916b0565d412c78e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:48.479213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-505822f9-e4ec-42ef-94e5-1035fc5c17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:48.488465 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1287/2580] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 10 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:48.499724 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d50710-d22a-420d-b17f-f3905bf42844 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:48.511675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d50710-d22a-420d-b17f-f3905bf42844 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:48.515219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:48.515810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.516085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.516350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.527679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:48.529110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:48.541417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1294/2581] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:53.402857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:53.403327 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:53.403431 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:53.422500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:53.423684 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:53.485166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1288/2582] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 10 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:53.508932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:53.510417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:53.582616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801b188-9edb-45fa-9866-cdefb62a263e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:53.588400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54ba59-738f-44a3-b123-7b8e1b17167c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:53.589099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54ba59-738f-44a3-b123-7b8e1b17167c service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:53.589382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54ba59-738f-44a3-b123-7b8e1b17167c service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:53.589659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54ba59-738f-44a3-b123-7b8e1b17167c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:53.605446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54ba59-738f-44a3-b123-7b8e1b17167c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:53.605446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54ba59-738f-44a3-b123-7b8e1b17167c service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:53.629394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1295/2583] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:32:55.140814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:55.197063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:55.230807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faa3e8-b853-4900-85d8-db4857316018 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:55.236284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:55.237105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:55.237394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:55.237671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] PATH_INFO: `/v3/users/920820f894244bd2a2a422b503c4b47a` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:55.238833 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=920820f894244bd2a2a422b503c4b47a)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:55.241031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:55.434816 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920820f894244bd2a2a422b503c4b47a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:55.502615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920820f894244bd2a2a422b503c4b47a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:55.512576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920820f894244bd2a2a422b503c4b47a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:55.513064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920820f894244bd2a2a422b503c4b47a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:55.540482 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920820f894244bd2a2a422b503c4b47a'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:55.547819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1289/2584] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:55 2022] DELETE /identity/v3/users/920820f894244bd2a2a422b503c4b47a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:55.557783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:55.558467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:55.558467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:55.579267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:55.580478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:55.652387 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1296/2585] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:55.664109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:55.665766 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:55.741780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260e1fa-94fc-46b0-adde-e15b3d01ef3f admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:55.746251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e9cb3-2136-4a05-a5f7-9dc993e3ce4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:55.746933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9cb3-2136-4a05-a5f7-9dc993e3ce4b service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:55.747254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9cb3-2136-4a05-a5f7-9dc993e3ce4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:55.747531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9cb3-2136-4a05-a5f7-9dc993e3ce4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:55.758834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e9cb3-2136-4a05-a5f7-9dc993e3ce4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:55.760340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e9cb3-2136-4a05-a5f7-9dc993e3ce4b service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:55.777993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1290/2586] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:32:56.315023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:56.316580 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:56.332386 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b64e-f462-402c-8075-f0066bd5e324 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:56.344117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:56.345340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:56.345955 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:56.346497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] PATH_INFO: `/v3/projects/b7c352f08c514569aa3833ea7beb52b5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:56.359636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c352f08c514569aa3833ea7beb52b5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:56.361649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:56.466363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c352f08c514569aa3833ea7beb52b5', 'request_id': 'req-e97a9c79-0106-482d-b707-4987389da029'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:56.471350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:56.474589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c352f08c514569aa3833ea7beb52b5', 'request_id': 'req-e97a9c79-0106-482d-b707-4987389da029'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:56.481150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c352f08c514569aa3833ea7beb52b5', 'request_id': 'req-e97a9c79-0106-482d-b707-4987389da029'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:56.566340 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] The token cache is being invalidate because project b7c352f08c514569aa3833ea7beb52b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:56.567502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:56.568631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e97a9c79-0106-482d-b707-4987389da029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:56.570630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1297/2587] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:56 2022] DELETE /identity/v3/projects/b7c352f08c514569aa3833ea7beb52b5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:56.576670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:56.577564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:56.577564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:56.613868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:56.616347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:56.702308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:56.715511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1291/2588] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:56.725354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:56.725621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:56.725936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:56.747694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:56.750669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:56.757947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1298/2589] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:56.762477 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:56.775417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e6d38-efb2-4890-a707-64904ef06c6b None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:56.781171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:56.782457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:56.782457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:56.782457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:56.783240 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:32:56.783649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:56.786647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:56.804175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1292/2590] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:32:56.814472 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:56.825033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d70ec-1596-490a-badf-b2aebcb1c4ba None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:56.829957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:56.830562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:56.830842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:56.831110 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:56.831729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:56.834320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:56.934182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1299/2591] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:56 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:56.935099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:56.946392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efc7c9-992c-49fb-9ab4-d494e79836b8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.006311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.007171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.007535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.007890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.008597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.010751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.135420 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1293/2592] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:56 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:57.136484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.147841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2dacc-90e8-4cf7-aeee-22ae89fb2237 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.153364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.154062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.154334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.154599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.155243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.156834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.172116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1300/2593] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:57.173358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.186072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d5698-fecf-4c4c-95f9-bd48fd71341a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.191212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779d6a1-3554-4e79-bc02-c4cd59089125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.191814 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d6a1-3554-4e79-bc02-c4cd59089125 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.192094 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d6a1-3554-4e79-bc02-c4cd59089125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.192480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d6a1-3554-4e79-bc02-c4cd59089125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.193960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779d6a1-3554-4e79-bc02-c4cd59089125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.195594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779d6a1-3554-4e79-bc02-c4cd59089125 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.210340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1294/2594] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:57.214514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.225408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9549a5-dcd5-48e2-9691-3cfd7b3527f7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.230195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.230980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.231367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.231715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] PATH_INFO: `/v3/projects/4c78bade814744a7b09717ab67cc16dd/users/f455191669934ff381914afc86037bf1/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.248631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c78bade814744a7b09717ab67cc16dd, user_id=f455191669934ff381914afc86037bf1, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.251393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.343347 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1301/2595] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:57 2022] PUT /identity/v3/projects/4c78bade814744a7b09717ab67cc16dd/users/f455191669934ff381914afc86037bf1/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:57.354852 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.355328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.355627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.389837 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] MFA Rules not processed for user `f455191669934ff381914afc86037bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:57.391344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:57.475887 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1295/2596] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:57.477907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.559863 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce920c6-6ac1-4350-ae89-442fb92d4c72 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.565285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.565983 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.566368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.566917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.568089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.570418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.595879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.598234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:57.615663 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcbedb-ecf2-4ecd-b9f8-6d709588f6d5 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.625584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.626534 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.626973 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.627556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] PATH_INFO: `/v3/users/3747aa23d07e4c7290472edb6ef58e8d` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.629783 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3747aa23d07e4c7290472edb6ef58e8d)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.633478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.661174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1302/2597] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:32:57 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:32:57.665763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.738811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354981f-f1f9-4afb-90a9-dc842274a00d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.744406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.745212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.745494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.745765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.746394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.748334 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.758499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:57.816697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:57.819555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3747aa23d07e4c7290472edb6ef58e8d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:57.840251 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1303/2598] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:32:57 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:32:57.844092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.854033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacf73f-cf20-44b1-a229-c1ed7842e954 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.859095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.859565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.859864 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.860149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.860775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.862406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.872787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1304/2599] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:32:57.876737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:57.887194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3baae-e3e1-4186-8712-fe88ede6f0be admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:57.891025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3747aa23d07e4c7290472edb6ef58e8d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:57.903568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:57.908264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.908492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.908778 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] PATH_INFO: `/v3/projects/e79537e864b2438c9bc9bc6ad2886785/users/dd4ee291fbec46ea9a449c60b18d2858/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.910143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3747aa23d07e4c7290472edb6ef58e8d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:57.911391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3747aa23d07e4c7290472edb6ef58e8d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:57.933791 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79537e864b2438c9bc9bc6ad2886785, user_id=dd4ee291fbec46ea9a449c60b18d2858, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:57.937097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:57.958574 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac5bdca6-7528-4a83-919f-3b0afa7f6393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3747aa23d07e4c7290472edb6ef58e8d'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:57.966242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1296/2600] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:57 2022] DELETE /identity/v3/users/3747aa23d07e4c7290472edb6ef58e8d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:57.971150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:57.971508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:57.971844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:57.993536 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:57.994826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:58.024172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1305/2601] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:57 2022] PUT /identity/v3/projects/e79537e864b2438c9bc9bc6ad2886785/users/dd4ee291fbec46ea9a449c60b18d2858/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:32:58.025383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:58.027454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:58.078892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1297/2602] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:58.089871 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:58.094312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:58.136490 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb1780-b31c-4278-a269-6846c11bf6fc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:58.141756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:58.142604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:58.142889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:58.143164 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:58.143823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:58.145561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:58.165118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1306/2603] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:58.167561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:58.178427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ebf31-5905-4123-b536-bcbb7c27ed55 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:58.183718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82743562-7305-416b-bc62-054ed8517388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:58.184616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743562-7305-416b-bc62-054ed8517388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:58.185346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743562-7305-416b-bc62-054ed8517388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:58.185590 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32575afa-09f0-47de-90d6-a0401b08f60f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:58.186244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743562-7305-416b-bc62-054ed8517388 admin admin] PATH_INFO: `/v3/projects/e79537e864b2438c9bc9bc6ad2886785/users/dd4ee291fbec46ea9a449c60b18d2858/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:58.188376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743562-7305-416b-bc62-054ed8517388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79537e864b2438c9bc9bc6ad2886785, user_id=dd4ee291fbec46ea9a449c60b18d2858, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:58.191494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743562-7305-416b-bc62-054ed8517388 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:58.211782 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1307/2604] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:32:58 2022] PUT /identity/v3/projects/e79537e864b2438c9bc9bc6ad2886785/users/dd4ee291fbec46ea9a449c60b18d2858/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:58.222032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:58.222676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:58.223100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:58.259781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] MFA Rules not processed for user `dd4ee291fbec46ea9a449c60b18d2858`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:58.261185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:58.265955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:58.266568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:58.266851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:58.267116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:58.277710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:58.279291 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:58.301708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1298/2605] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:58.373113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1308/2606] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:32:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:58.381130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:58.383350 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:58.412653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:58.503801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6898f-b35f-404d-be05-8c7e8ae08551 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:58.568702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:58.569361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:58.569648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:58.569927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:58.589039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:58.589039 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:58.604683 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1299/2607] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:32:58.842895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:58.844393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:58.918662 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:58.990892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631f6a-6316-411d-8a3d-c0e382140933 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:59.003757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:59.004481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:59.006206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:59.006206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] PATH_INFO: `/v3/projects/b8d4047a7ffa4a95ae0b4cbf3c39ad4c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:59.009112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d4047a7ffa4a95ae0b4cbf3c39ad4c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:59.011212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:59.413592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d4047a7ffa4a95ae0b4cbf3c39ad4c', 'request_id': 'req-29fc2321-27d1-49c6-9b6e-390b4332fb2e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.419947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d4047a7ffa4a95ae0b4cbf3c39ad4c', 'request_id': 'req-29fc2321-27d1-49c6-9b6e-390b4332fb2e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.432929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d4047a7ffa4a95ae0b4cbf3c39ad4c', 'request_id': 'req-29fc2321-27d1-49c6-9b6e-390b4332fb2e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.510548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] The token cache is being invalidate because project b8d4047a7ffa4a95ae0b4cbf3c39ad4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:32:59.510995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.511755 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.513683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1309/2608] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:32:58 2022] DELETE /identity/v3/projects/b8d4047a7ffa4a95ae0b4cbf3c39ad4c => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:32:59.518005 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:59.530434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca14fc-649a-4426-86ee-015c64ec3393 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:59.620433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:59.623246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:59.623246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:59.623246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] PATH_INFO: `/v3/users/64ba3341f0f1497c88e391a43217b781` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:59.623246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ba3341f0f1497c88e391a43217b781)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:59.624846 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:59.879573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ba3341f0f1497c88e391a43217b781'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.954838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:59.954838 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:59.972345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ba3341f0f1497c88e391a43217b781'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.980867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ba3341f0f1497c88e391a43217b781'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:32:59.981617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ba3341f0f1497c88e391a43217b781'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.007066 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ba3341f0f1497c88e391a43217b781'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.009034 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:00.014749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1300/2609] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:32:59 2022] DELETE /identity/v3/users/64ba3341f0f1497c88e391a43217b781 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:00.025792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc2321-27d1-49c6-9b6e-390b4332fb2e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:00.097702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd4ee291fbec46ea9a449c60b18d2858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e79537e864b2438c9bc9bc6ad2886785', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:00.098323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:00.098727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:00.099018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:00.099776 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:00.100081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:00.101711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:00.109995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:00.119733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1310/2610] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:33:00.121340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:00.122906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:00.140584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350896-e61b-43b5-a781-02f113f7491d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:00.146284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd4ee291fbec46ea9a449c60b18d2858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e79537e864b2438c9bc9bc6ad2886785', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:00.150095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:00.150095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:00.150095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:00.150095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:00.150610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:00.265091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1301/2611] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:00 2022] POST /identity/v3/projects => generated 353 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:00.269224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:00.281747 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618769fb-47d9-44a3-b11a-fe99e4685ece tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:00.349250 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd4ee291fbec46ea9a449c60b18d2858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e79537e864b2438c9bc9bc6ad2886785', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:00.349888 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:00.350172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:00.350454 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:00.351114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:00.351410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:00.353019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:00.365217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1311/2612] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:33:00.402227 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:00.413546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c8231-80a5-4a78-bff7-e514742618db tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:00.418191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd4ee291fbec46ea9a449c60b18d2858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e79537e864b2438c9bc9bc6ad2886785', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:00.419823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:00.419823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:00.419823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] PATH_INFO: `/v3/projects/07a555ec12d742ad8359c2472185a98c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:00.420520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorizing `identity:delete_project(project_id=07a555ec12d742ad8359c2472185a98c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:00.422485 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:00.499687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:00.505521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:00.586278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a555ec12d742ad8359c2472185a98c', 'request_id': 'req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.597288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a555ec12d742ad8359c2472185a98c', 'request_id': 'req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.606881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a555ec12d742ad8359c2472185a98c', 'request_id': 'req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.621502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476851b-ca35-47ac-934f-e092d7f3e629 tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:00.675047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] The token cache is being invalidate because project 07a555ec12d742ad8359c2472185a98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:00.675810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.676049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3e92bac6-6294-40c1-b4e1-3137ac15b6ef tempest-VolumeQuotaClassesTest-867832270 tempest-VolumeQuotaClassesTest-867832270-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.679438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1302/2613] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:00 2022] DELETE /identity/v3/projects/07a555ec12d742ad8359c2472185a98c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:00.706375 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:00.707277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:00.707460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:00.707732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] PATH_INFO: `/v3/projects/dd52a0d027604a40a29dedb1923ed040` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:00.709020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd52a0d027604a40a29dedb1923ed040)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:00.710909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:00.848122 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:00.857031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd52a0d027604a40a29dedb1923ed040', 'request_id': 'req-acfb8d23-e722-4457-ab76-6c3d581e0f99'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.878452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd52a0d027604a40a29dedb1923ed040', 'request_id': 'req-acfb8d23-e722-4457-ab76-6c3d581e0f99'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.893353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd52a0d027604a40a29dedb1923ed040', 'request_id': 'req-acfb8d23-e722-4457-ab76-6c3d581e0f99'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.983115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] The token cache is being invalidate because project dd52a0d027604a40a29dedb1923ed040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:00.983725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.984269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-acfb8d23-e722-4457-ab76-6c3d581e0f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:00.986096 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1312/2614] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:00 2022] DELETE /identity/v3/projects/dd52a0d027604a40a29dedb1923ed040 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:00.996339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:00.997050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:00.997526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.019665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:01.021860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.105062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1303/2615] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:01.114054 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.114462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.114736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.135371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:01.136604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.144865 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1313/2616] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:01.150010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.163454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b7a78-50af-4e69-8ad2-1cbd5b9f2c09 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.169475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.170182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.170567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.170951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.171828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:01.177394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.179087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.191337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1304/2617] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:33:01.196678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.206167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a390f-f39a-4431-8a30-3370d6acc5c0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.212045 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.212712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.213006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.213291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.213997 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.216028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.362608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1314/2618] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:01 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:01.367614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.378424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c90139-cdbe-46f4-a80c-3c23797fee6a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.432899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:01.435725 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.437816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.446666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.447523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.447903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.448279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.449038 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.451192 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.528081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668afd6a-a5f4-4181-bc49-b2a049f7b620 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.532848 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.533515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.533812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.534329 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] PATH_INFO: `/v3/users/f455191669934ff381914afc86037bf1` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.535469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f455191669934ff381914afc86037bf1)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.538003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.585385 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1305/2619] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:01 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:01.589287 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.601234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388277e-7317-4c2a-b533-c5257c9c0eaf admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.622978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.622978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.622978 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.623585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.623811 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.625497 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.639023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1306/2620] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:01.644743 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.662451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a313af-99ec-4e1e-bd22-b795ffee0d7e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.669373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.669989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.670271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.670545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.671191 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.672838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.686648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1307/2621] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:01.691424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.704633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a334b-57cf-4d7f-836c-a396f8f8a2ff admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.710649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd55a24-5b52-4bbc-9588-f29641836ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.711400 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd55a24-5b52-4bbc-9588-f29641836ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.711745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd55a24-5b52-4bbc-9588-f29641836ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.712103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd55a24-5b52-4bbc-9588-f29641836ea7 admin admin] PATH_INFO: `/v3/projects/a3331d8d955e4847b368a68ca12d28d9/users/ce3d93f41cac4b9c97fdf3d623f5e771/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.729661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd55a24-5b52-4bbc-9588-f29641836ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3331d8d955e4847b368a68ca12d28d9, user_id=ce3d93f41cac4b9c97fdf3d623f5e771, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.732576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd55a24-5b52-4bbc-9588-f29641836ea7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:01.743768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f455191669934ff381914afc86037bf1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:01.752942 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f455191669934ff381914afc86037bf1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:01.825957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1308/2622] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:01 2022] PUT /identity/v3/projects/a3331d8d955e4847b368a68ca12d28d9/users/ce3d93f41cac4b9c97fdf3d623f5e771/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:01.836623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f455191669934ff381914afc86037bf1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:01.837514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f455191669934ff381914afc86037bf1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:01.837849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.838208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.838470 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.864184 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3c691de9-36aa-4a12-814c-f04969561994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f455191669934ff381914afc86037bf1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:01.869203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1315/2623] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:01 2022] DELETE /identity/v3/users/f455191669934ff381914afc86037bf1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:01.873684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.873926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.874203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.874970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] MFA Rules not processed for user `ce3d93f41cac4b9c97fdf3d623f5e771`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:01.876249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.894086 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:01.895351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.909550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:01.960073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1309/2624] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:01.964223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.966569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.976052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1316/2625] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:01.982082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2429f9b-57fc-4010-bc0c-fb203a3e2a3a None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:01.984299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:01.986762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:01.990377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:01.990377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:01.990377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:01.990377 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:01.991013 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:01.993708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.081299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822569c-74e0-413d-b4b5-fefd03e72c6e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.089482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.090270 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.090676 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.091061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.101229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.102727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.113194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1317/2626] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:02.354979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1310/2627] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:01 2022] POST /identity/v3/projects => generated 383 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:02.356112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.366209 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01436-3076-4ddc-b806-3dae8dd0088e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.426093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.426815 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.427079 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.427323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.427952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.429901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.519614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1318/2628] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:02 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:02.529561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.542223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ecacb-d93c-425b-acb0-5f676bdfbc11 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.547087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.547671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.547953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.548370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.549058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.556207 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.569115 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1311/2629] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:02.570092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.582451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233fee5-7be1-4612-8abb-218f1e1dc30f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.587161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.587825 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.588107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.588385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] PATH_INFO: `/v3/projects/2ca6807204a440bfb3a5b2348e3afe01/users/4a5e3715130e4f59b308f760e9847914/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.604354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca6807204a440bfb3a5b2348e3afe01, user_id=4a5e3715130e4f59b308f760e9847914, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.607075 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.675372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1319/2630] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:02 2022] PUT /identity/v3/projects/2ca6807204a440bfb3a5b2348e3afe01/users/4a5e3715130e4f59b308f760e9847914/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:02.679548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.689569 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4fdde-2e7c-430e-ad8e-6c6eff7951cb admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.721444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.725634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:02.738621 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:02.756311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.757085 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.757373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.757651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.758308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.759909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.766602 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:02.770769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1312/2631] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:02.774501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.785481 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3674222-f1c2-44df-b264-21dd52efb572 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.791828 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5625-1696-4826-a0c0-981f9e79a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.792719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5625-1696-4826-a0c0-981f9e79a2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.793198 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5625-1696-4826-a0c0-981f9e79a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.793652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5625-1696-4826-a0c0-981f9e79a2f7 admin admin] PATH_INFO: `/v3/projects/2ca6807204a440bfb3a5b2348e3afe01/users/4a5e3715130e4f59b308f760e9847914/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.795656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f5625-1696-4826-a0c0-981f9e79a2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca6807204a440bfb3a5b2348e3afe01, user_id=4a5e3715130e4f59b308f760e9847914, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.798342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f5625-1696-4826-a0c0-981f9e79a2f7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.801037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7155b5b-24a5-4273-9ddb-6b8b28a374e9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:02.806420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:02.807085 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.807369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.807669 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] PATH_INFO: `/v3/projects/4c78bade814744a7b09717ab67cc16dd` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.808830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c78bade814744a7b09717ab67cc16dd)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:02.810759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:02.812506 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1313/2632] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:02 2022] PUT /identity/v3/projects/2ca6807204a440bfb3a5b2348e3afe01/users/4a5e3715130e4f59b308f760e9847914/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:02.827893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:02.828184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:02.828463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:02.856188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] MFA Rules not processed for user `4a5e3715130e4f59b308f760e9847914`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:02.857451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:02.921533 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c78bade814744a7b09717ab67cc16dd', 'request_id': 'req-0a52404e-75d0-46de-a1d8-ac3357938e8e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:02.926819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1314/2633] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:02.927641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c78bade814744a7b09717ab67cc16dd', 'request_id': 'req-0a52404e-75d0-46de-a1d8-ac3357938e8e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:02.934333 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c78bade814744a7b09717ab67cc16dd', 'request_id': 'req-0a52404e-75d0-46de-a1d8-ac3357938e8e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:02.938228 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:02.939673 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:02.997203 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318bcc3-dd5d-4eb1-83e8-d29ab1ef85ef None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:03.006117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] The token cache is being invalidate because project 4c78bade814744a7b09717ab67cc16dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:03.006657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.007105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.008756 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1320/2634] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:02 2022] DELETE /identity/v3/projects/4c78bade814744a7b09717ab67cc16dd => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:03.012626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:03.014145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:03.049179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:03.049794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:03.050079 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:03.050349 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:03.052414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:03.078417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:03.078856 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52404e-75d0-46de-a1d8-ac3357938e8e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:03.084173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:03.084803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:03.085115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:03.085389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] PATH_INFO: `/v3/users/dd4ee291fbec46ea9a449c60b18d2858` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:03.086746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4ee291fbec46ea9a449c60b18d2858)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:03.088616 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:03.120049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:03.121545 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:03.131678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1315/2635] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:03.271668 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4ee291fbec46ea9a449c60b18d2858'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.278358 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4ee291fbec46ea9a449c60b18d2858'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.281827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:03.306518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:03.307959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:03.319296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfaff0-c44b-4248-b6ed-29e6aa504e23 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:03.322948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:03.324072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:03.324072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:03.324072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:03.325784 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:03.343093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4ee291fbec46ea9a449c60b18d2858'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.344232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4ee291fbec46ea9a449c60b18d2858'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.357246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4ee291fbec46ea9a449c60b18d2858'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:03.363029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1321/2636] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:03 2022] DELETE /identity/v3/users/dd4ee291fbec46ea9a449c60b18d2858 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:03.386199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:03.387631 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:03.397546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1316/2637] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:03.854591 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:03.858356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:03.887340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:03.887340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:03.900601 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a90a1-545b-4d14-8a5e-a81eabc748ee service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:03.929762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f6e2-4f66-417a-b953-a6cadeac4c9b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:03.934702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:03.935589 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:03.935876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:03.936971 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] PATH_INFO: `/v3/projects/e79537e864b2438c9bc9bc6ad2886785` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:03.938230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e79537e864b2438c9bc9bc6ad2886785)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:03.940141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:03.960071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:03.960910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:03.961293 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:03.961568 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] PATH_INFO: `/v3/users/ce3d93f41cac4b9c97fdf3d623f5e771` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:03.962753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce3d93f41cac4b9c97fdf3d623f5e771)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:03.964692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.058728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e79537e864b2438c9bc9bc6ad2886785', 'request_id': 'req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.064121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e79537e864b2438c9bc9bc6ad2886785', 'request_id': 'req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.071202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e79537e864b2438c9bc9bc6ad2886785', 'request_id': 'req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.142356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] The token cache is being invalidate because project e79537e864b2438c9bc9bc6ad2886785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:04.142654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.143281 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4e1f9d76-8ea2-4ebd-875f-5898ad9d1d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.144796 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1322/2638] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:03 2022] DELETE /identity/v3/projects/e79537e864b2438c9bc9bc6ad2886785 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:04.150398 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6f458-a543-4a18-ba20-071e1f6fa559 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.150685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6f458-a543-4a18-ba20-071e1f6fa559 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.151059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6f458-a543-4a18-ba20-071e1f6fa559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.151754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce3d93f41cac4b9c97fdf3d623f5e771'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.170935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8fb6f458-a543-4a18-ba20-071e1f6fa559 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:04.172126 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8fb6f458-a543-4a18-ba20-071e1f6fa559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:04.227808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce3d93f41cac4b9c97fdf3d623f5e771'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.229397 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1323/2639] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:04.233632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:04.236062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce3d93f41cac4b9c97fdf3d623f5e771'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.237241 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce3d93f41cac4b9c97fdf3d623f5e771'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.238389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.238692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.239709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.250213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ac9e8617-7059-4b9a-b9f0-7b46a1361c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce3d93f41cac4b9c97fdf3d623f5e771'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:04.257345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1317/2640] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:03 2022] DELETE /identity/v3/users/ce3d93f41cac4b9c97fdf3d623f5e771 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:04.261667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.261965 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.262179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:04.262310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.263389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:04.267701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:04.280754 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:04.282020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:04.332470 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1324/2641] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:04.336760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.345516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd8867-68a1-4f57-9795-82eec4588761 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.350954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.351564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.351840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.352109 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.352316 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1318/2642] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:04.357819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:04.357819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.357819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.359439 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.361513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:04.368318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1325/2643] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:33:04.372434 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.382835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b81c-7b87-47ae-bab4-238940e021f7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.388332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.389194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.389354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.389548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.390354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.392052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.427134 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25583f1d-858d-4bac-abf6-ee9303caccf2 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.430890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.431498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.431780 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.432054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.443017 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.444810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.455483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1319/2644] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:04.510736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1326/2645] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:04 2022] POST /identity/v3/projects => generated 389 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:04.515006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.525358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ed8e-fced-4275-aeac-f1f8ea6bcb05 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.586753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.587280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.587619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.587900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.588530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.590518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.689047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1320/2646] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:04 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:04.694996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.705246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f9988-d2f4-4461-8d31-46039cf416bc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.710005 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.710753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.711033 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.711342 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.711950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.713577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.724542 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1327/2647] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:04.728733 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.739007 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0d357-b800-4e62-a35c-dd363cf57432 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.744903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.745639 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.745922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.746194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.746834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.748741 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.754076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:04.759972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1321/2648] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:04.769409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.780253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c210bd-0784-4122-b01e-20dd54128c01 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.785481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412ccea-ccf8-4385-aa12-c14387c08744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:04.786631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412ccea-ccf8-4385-aa12-c14387c08744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.787179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412ccea-ccf8-4385-aa12-c14387c08744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.787465 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412ccea-ccf8-4385-aa12-c14387c08744 admin admin] PATH_INFO: `/v3/projects/73f4bbae3f1b41b29e8d2d215a2a81d9/users/b38a98cbcd8b451faa7b414499fcee29/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.805913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412ccea-ccf8-4385-aa12-c14387c08744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f4bbae3f1b41b29e8d2d215a2a81d9, user_id=b38a98cbcd8b451faa7b414499fcee29, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:04.808683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412ccea-ccf8-4385-aa12-c14387c08744 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:04.900333 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:04.904209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:04.923810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770088a2-fda9-4c13-86bf-f885032458a4 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:04.932444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1328/2649] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:04 2022] PUT /identity/v3/projects/73f4bbae3f1b41b29e8d2d215a2a81d9/users/b38a98cbcd8b451faa7b414499fcee29/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:04.943595 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:04.944654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:04.945318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:04.983025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] MFA Rules not processed for user `b38a98cbcd8b451faa7b414499fcee29`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:04.984587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:05.031715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.032348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.032632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.032903 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] PATH_INFO: `/v3/projects/a3331d8d955e4847b368a68ca12d28d9` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.034336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3331d8d955e4847b368a68ca12d28d9)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.036394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.077657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1329/2650] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:05.082195 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.092664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef09fca-87cf-4106-9f0a-0feb06dd5d52 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.097651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.097800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.098019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.098290 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.099548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.101449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.171906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3331d8d955e4847b368a68ca12d28d9', 'request_id': 'req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.177899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3331d8d955e4847b368a68ca12d28d9', 'request_id': 'req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.183979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3331d8d955e4847b368a68ca12d28d9', 'request_id': 'req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.190507 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1330/2651] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:05 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:05.194441 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.202840 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:05.258464 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3b41b-6a66-46ec-a2a5-d4d7a1036a91 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.263114 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.263714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.263992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.264264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.264884 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.266818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.284637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] The token cache is being invalidate because project a3331d8d955e4847b368a68ca12d28d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:05.284934 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.285597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.287806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1322/2652] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:04 2022] DELETE /identity/v3/projects/a3331d8d955e4847b368a68ca12d28d9 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:05.291642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.292929 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:05.356768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b09a6-9839-4003-b8d3-0ebc8d8d4cf5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.361457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.362073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.362354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.362623 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] PATH_INFO: `/v3/users/4a5e3715130e4f59b308f760e9847914` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.363734 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a5e3715130e4f59b308f760e9847914)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.366209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.382980 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1331/2653] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:05 2022] POST /identity/v3/users => generated 301 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:05.386614 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.388202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:05.400579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643bfc9-9066-4213-aecf-10968831c914 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.405592 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.406179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.406459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.419922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.419922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.419922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.419922 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1332/2654] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:05.423451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.432911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03194200-7e8c-4b85-9ec6-f1bd20c8eaab admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.437556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.438244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.438526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.438803 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] PATH_INFO: `/v3/projects/102b380283f1462f8bc95723add091de/users/1969a03736d44b9fa0a1d0383f8794a1/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.454172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102b380283f1462f8bc95723add091de, user_id=1969a03736d44b9fa0a1d0383f8794a1, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.456901 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.530150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1333/2655] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:05 2022] PUT /identity/v3/projects/102b380283f1462f8bc95723add091de/users/1969a03736d44b9fa0a1d0383f8794a1/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:05.534258 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.555185 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:05.566549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a5e3715130e4f59b308f760e9847914'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.595173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11333-28e8-4bfb-8a42-a37f41a01e15 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.600296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.600931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.601234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.601503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.602140 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.603708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.613447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1334/2656] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:05.617867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.627091 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985c8b-3d8e-486f-88f7-0087cd98a95b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.632369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77132a9f-deb2-48a7-b2fa-87b571097c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.633012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132a9f-deb2-48a7-b2fa-87b571097c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.633296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132a9f-deb2-48a7-b2fa-87b571097c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.633567 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132a9f-deb2-48a7-b2fa-87b571097c2a admin admin] PATH_INFO: `/v3/projects/102b380283f1462f8bc95723add091de/users/1969a03736d44b9fa0a1d0383f8794a1/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.635451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77132a9f-deb2-48a7-b2fa-87b571097c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=102b380283f1462f8bc95723add091de, user_id=1969a03736d44b9fa0a1d0383f8794a1, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.638064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77132a9f-deb2-48a7-b2fa-87b571097c2a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.640167 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a5e3715130e4f59b308f760e9847914'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.652444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1335/2657] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:05 2022] PUT /identity/v3/projects/102b380283f1462f8bc95723add091de/users/1969a03736d44b9fa0a1d0383f8794a1/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:05.653312 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a5e3715130e4f59b308f760e9847914'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.654572 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a5e3715130e4f59b308f760e9847914'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.659170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.659463 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.659741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.669760 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a5e3715130e4f59b308f760e9847914'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:05.675767 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1323/2658] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:05 2022] DELETE /identity/v3/users/4a5e3715130e4f59b308f760e9847914 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:05.691112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] MFA Rules not processed for user `1969a03736d44b9fa0a1d0383f8794a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:05.697227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:05.726469 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:05.764664 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1336/2659] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:05.777213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:05.778642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:05.837352 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d0d3-31b9-42d7-87e6-dae41df639e6 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:05.841083 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:05.841694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:05.841979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:05.842253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:05.852512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:05.853999 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:05.867354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1324/2660] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:06.019844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:06.022914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:06.090432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d29cc-a9b6-43b7-abf8-d241af0a5a09 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:06.095119 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:06.095749 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.096029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.096294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] PATH_INFO: `/v3/users/b38a98cbcd8b451faa7b414499fcee29` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.097424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38a98cbcd8b451faa7b414499fcee29)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:06.099954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:06.129851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:06.131645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:06.146709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708885d-c30a-47c3-b8ae-75b5aa315780 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:06.151880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:06.152524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.152824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.153114 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] PATH_INFO: `/v3/projects/2ca6807204a440bfb3a5b2348e3afe01` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.154808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca6807204a440bfb3a5b2348e3afe01)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:06.156260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:06.305386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca6807204a440bfb3a5b2348e3afe01', 'request_id': 'req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.307044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38a98cbcd8b451faa7b414499fcee29'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.312026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca6807204a440bfb3a5b2348e3afe01', 'request_id': 'req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.320376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38a98cbcd8b451faa7b414499fcee29'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.320544 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca6807204a440bfb3a5b2348e3afe01', 'request_id': 'req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.389061 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38a98cbcd8b451faa7b414499fcee29'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.389857 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38a98cbcd8b451faa7b414499fcee29'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.403792 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8833fe47-04d0-435f-9aa6-02d16bae082a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38a98cbcd8b451faa7b414499fcee29'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.410808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1337/2661] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:06 2022] DELETE /identity/v3/users/b38a98cbcd8b451faa7b414499fcee29 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:06.414961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.415320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.415598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.436484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:06.437714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:06.457875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] The token cache is being invalidate because project 2ca6807204a440bfb3a5b2348e3afe01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:06.458220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.459525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-fe08c5e3-6ff8-4434-8a8e-4fbda2ccc616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:06.461795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1325/2662] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:06 2022] DELETE /identity/v3/projects/2ca6807204a440bfb3a5b2348e3afe01 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:06.467194 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d4e26-8f27-4d26-9b50-290f4317ae2b None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.468143 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d4e26-8f27-4d26-9b50-290f4317ae2b None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.468423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d4e26-8f27-4d26-9b50-290f4317ae2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.492376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:06.493856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-926d4e26-8f27-4d26-9b50-290f4317ae2b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:06.495084 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-926d4e26-8f27-4d26-9b50-290f4317ae2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:06.510442 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1338/2663] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:06.521867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:06.523626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:06.535596 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:06.572178 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1326/2664] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:06.580045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.580239 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.580827 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.600179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9088e-af15-4674-a2ca-7260dbd00bb1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:06.603233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:06.604151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:06.604484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:06.604802 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.605104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.605427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.613386 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1327/2665] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:06.617315 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:06.618072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:06.619674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:06.627962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a6722-4cdb-4dca-b83b-d6eb4f2a56be None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:06.631710 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1339/2666] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:06.633091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:06.633710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.633993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.634266 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.635037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:06.635347 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:06.636963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:06.662308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1328/2667] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:33:06.666468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:06.679427 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea5e2e-f43a-4769-a8ce-43b59184747f service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:06.684211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:06.684811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.685193 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.685618 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.686023 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:06.687920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:06.814513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1340/2668] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:06 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:06.818234 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:06.829040 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966587-412b-4488-aa26-56e0afb9a87b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:06.901636 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:06.902416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:06.902667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:06.902928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:06.903613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:06.905605 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:07.006671 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1329/2669] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:06 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:07.010881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:07.021233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ef7-3c39-41e6-b20a-3042d8815342 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:07.025851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:07.026447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.026728 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.027000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.027638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:07.029417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:07.040187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1341/2670] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:07.049277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:07.059399 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6be508-fd6b-420d-9d2d-4bf50634134d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:07.064057 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:07.064644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.064926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.065231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.065869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:07.068091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:07.079019 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1330/2671] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:07.081605 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:07.083706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:07.084361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:07.098543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cbaeb-3d33-4622-996b-efa9f396c97e admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:07.098851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27be3e-d30b-4de5-8a2c-77ac729f6997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:07.099513 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27be3e-d30b-4de5-8a2c-77ac729f6997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.099795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27be3e-d30b-4de5-8a2c-77ac729f6997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.100067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27be3e-d30b-4de5-8a2c-77ac729f6997 admin admin] PATH_INFO: `/v3/projects/9454b3039aac4bc4a4d4881bd23cf55b/users/efd30a8453144a1b97d8fff6ea745e68/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.109379 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a132a-c390-4baf-98cc-f596fe5ad68a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:07.115125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27be3e-d30b-4de5-8a2c-77ac729f6997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9454b3039aac4bc4a4d4881bd23cf55b, user_id=efd30a8453144a1b97d8fff6ea745e68, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:07.115420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:07.116062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.116345 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.116713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] PATH_INFO: `/v3/projects/73f4bbae3f1b41b29e8d2d215a2a81d9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.117796 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27be3e-d30b-4de5-8a2c-77ac729f6997 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:07.117911 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f4bbae3f1b41b29e8d2d215a2a81d9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:07.119748 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:07.184458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1331/2672] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:07 2022] PUT /identity/v3/projects/9454b3039aac4bc4a4d4881bd23cf55b/users/efd30a8453144a1b97d8fff6ea745e68/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:07.195405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.195815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.196092 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.227345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] MFA Rules not processed for user `efd30a8453144a1b97d8fff6ea745e68`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:07.228584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:07.233235 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f4bbae3f1b41b29e8d2d215a2a81d9', 'request_id': 'req-d3320716-2c54-4ffa-b122-a7db546f768e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:07.240986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f4bbae3f1b41b29e8d2d215a2a81d9', 'request_id': 'req-d3320716-2c54-4ffa-b122-a7db546f768e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:07.319762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f4bbae3f1b41b29e8d2d215a2a81d9', 'request_id': 'req-d3320716-2c54-4ffa-b122-a7db546f768e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:07.320727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:07.320727 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1332/2673] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:07.335037 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:07.337237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:07.401116 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161050-1acf-4441-a664-501bbabfd436 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:07.404959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:07.405787 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.406099 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.406406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.418515 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:07.420951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:07.432360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1333/2674] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:07.452424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:07.652272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35417e98-0069-4516-9ec6-f3c94ee02130 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:07.656957 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:07.657650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:07.657939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:07.658221 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] PATH_INFO: `/v3/users/efd30a8453144a1b97d8fff6ea745e68` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:07.659687 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd30a8453144a1b97d8fff6ea745e68)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:07.661762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:08.074088 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] The token cache is being invalidate because project 73f4bbae3f1b41b29e8d2d215a2a81d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:08.074429 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.075012 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.076481 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1342/2675] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:07 2022] DELETE /identity/v3/projects/73f4bbae3f1b41b29e8d2d215a2a81d9 => generated 0 bytes in 995 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:08.080304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:08.081719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:08.095685 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3320716-2c54-4ffa-b122-a7db546f768e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:08.101624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:08.102498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:08.103116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:08.103222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] PATH_INFO: `/v3/users/1969a03736d44b9fa0a1d0383f8794a1` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:08.104535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1969a03736d44b9fa0a1d0383f8794a1)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:08.106503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:08.195985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd30a8453144a1b97d8fff6ea745e68'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.263044 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd30a8453144a1b97d8fff6ea745e68'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.269458 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:08.274507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd30a8453144a1b97d8fff6ea745e68'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.274507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd30a8453144a1b97d8fff6ea745e68'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.278372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1969a03736d44b9fa0a1d0383f8794a1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.284753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-1ac66a50-f028-494d-a5c5-3f0b349b9847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd30a8453144a1b97d8fff6ea745e68'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.285485 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1969a03736d44b9fa0a1d0383f8794a1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.291640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1334/2676] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:07 2022] DELETE /identity/v3/users/efd30a8453144a1b97d8fff6ea745e68 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:08.295910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:08.296260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:08.296555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:08.315346 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:08.316531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:08.333388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:08.372938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1335/2677] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:08.381437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1969a03736d44b9fa0a1d0383f8794a1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.381437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1969a03736d44b9fa0a1d0383f8794a1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.383943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:08.385443 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:08.392926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fb42bb1f-635e-46d9-b3a6-600691902129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1969a03736d44b9fa0a1d0383f8794a1'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:08.399404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1343/2678] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:08 2022] DELETE /identity/v3/users/1969a03736d44b9fa0a1d0383f8794a1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:08.450747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd0679-d320-4036-8685-81025f1bb9ab None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:08.518554 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:08.519543 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:08.519952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:08.521032 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:08.521576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:08.521982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:08.522303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:08.524672 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:08.546594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] MFA Rules not processed for user `d0559bf2dae143468a9d25c82770a337`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:08.548962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:08.605028 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:08.606498 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:08.612129 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1344/2679] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 10 13:33:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:08.617151 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1336/2680] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:08.625450 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:08.627080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:08.721513 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc84d57-bed9-4680-8df4-2a7cddc11884 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:08.728904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bb11e8a7a124630bf9b0e611324fabf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:08.729373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:08.729637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:08.729910 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:08.742179 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:08.749950 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:08.754243 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1345/2681] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 10 13:33:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:33:08.992086 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:08.993652 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.005628 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f57a-5364-4ba8-b0b7-eec32ac87e9c service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.010430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.011077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.011413 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.011617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] PATH_INFO: `/v3/projects/102b380283f1462f8bc95723add091de` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.012753 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=102b380283f1462f8bc95723add091de)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.014715 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.043255 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:09.196826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.198359 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.213511 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89890be3-9184-40c3-be54-d9734d1c407c service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.232130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102b380283f1462f8bc95723add091de', 'request_id': 'req-c069e826-599d-4cb5-9b3d-2dd336a67211'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.242153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102b380283f1462f8bc95723add091de', 'request_id': 'req-c069e826-599d-4cb5-9b3d-2dd336a67211'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.248501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102b380283f1462f8bc95723add091de', 'request_id': 'req-c069e826-599d-4cb5-9b3d-2dd336a67211'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.264049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:09.275638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.276264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.276540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.276810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] PATH_INFO: `/v3/projects/9454b3039aac4bc4a4d4881bd23cf55b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.278004 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9454b3039aac4bc4a4d4881bd23cf55b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.279887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.353724 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] The token cache is being invalidate because project 102b380283f1462f8bc95723add091de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:09.354022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.357711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c069e826-599d-4cb5-9b3d-2dd336a67211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.357711 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1337/2682] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:08 2022] DELETE /identity/v3/projects/102b380283f1462f8bc95723add091de => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:09.360927 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe27d5-1bdc-4f00-8539-d7e28b0f34cf None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.361245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe27d5-1bdc-4f00-8539-d7e28b0f34cf None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.361521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe27d5-1bdc-4f00-8539-d7e28b0f34cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.382560 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7dfe27d5-1bdc-4f00-8539-d7e28b0f34cf None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:09.383899 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7dfe27d5-1bdc-4f00-8539-d7e28b0f34cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.393754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9454b3039aac4bc4a4d4881bd23cf55b', 'request_id': 'req-cac6d108-0a54-4da1-8c64-702d14eb2e71'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.399746 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9454b3039aac4bc4a4d4881bd23cf55b', 'request_id': 'req-cac6d108-0a54-4da1-8c64-702d14eb2e71'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.406100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9454b3039aac4bc4a4d4881bd23cf55b', 'request_id': 'req-cac6d108-0a54-4da1-8c64-702d14eb2e71'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.446691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1338/2683] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:09.454415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.455305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.455305 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.476247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:09.477502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.481062 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] The token cache is being invalidate because project 9454b3039aac4bc4a4d4881bd23cf55b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:09.484185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.484185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cac6d108-0a54-4da1-8c64-702d14eb2e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:09.484185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1346/2684] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:09 2022] DELETE /identity/v3/projects/9454b3039aac4bc4a4d4881bd23cf55b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:09.485710 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1339/2685] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:09.488144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.488430 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.488770 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.489844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.499875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3be84-294a-4fa3-84bb-8c252ffd69ee None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.504571 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.505188 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.505539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.506001 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.506774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:09.507076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.508931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32364-e524-4768-9756-89bbd7805c22 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.509452 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:09.510656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.519394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1347/2686] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:09.525212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1340/2687] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:33:09.528305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.529752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.530483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.530778 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.531050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.546761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329062da-1e0d-4204-8397-dbc26c303d3b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.551655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:09.553035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:09.553627 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.554784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.555280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.555754 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.556929 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.560038 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.562103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1341/2688] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:09.566369 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.576412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12404c5c-27a8-4218-9107-9457e7cf9579 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.581757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.582342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.582618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.582885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.584025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:09.584331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.586050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.597591 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1342/2689] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:33:09.606425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.617454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301684d-6e21-4108-9d7e-ed6db09514b4 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.622580 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.623179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.623462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.623735 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.624380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.626272 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.679264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1348/2690] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:09.683346 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.723281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1343/2691] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:09.729608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.758947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c4782-dc78-488d-b607-4e9db058e1f3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.793444 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:09.799880 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5bff-b6f9-44b3-8fa7-277216af542c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.810101 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.810716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.810993 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.811262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.811928 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.814033 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.818484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:09.824525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.825026 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.825473 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.825687 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.826170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.828076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.887882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1344/2692] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:09.892423 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.900502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1349/2693] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:09 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:09.904681 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.906374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033024c0-5005-45d3-8f75-408cd8f3b143 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.911504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.912776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.912776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.912776 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.913603 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.915525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.916696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84579707-de40-466d-9b63-5b0785fabe4c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.922001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.922736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.922905 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.923174 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.923813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.925501 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.927206 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1345/2694] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:09.930173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.937409 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1350/2695] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:09.941475 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.941629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921bc41-6e8f-4d4f-a7de-9901fd3e6792 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.947230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.947980 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.950045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.950045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.950045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.950838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.952175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289dcdc-c06e-4706-9b13-948c033c13a7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.956953 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.957563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.957841 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.958111 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.959626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:09.972088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1346/2696] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:09.972253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:09.972397 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.975733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1351/2697] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:09.977135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbb9e-71f1-47be-9f78-78c699d13afe admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.978147 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:09.982146 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb4d37-178d-4427-9216-1da4f7ee7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.982826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb4d37-178d-4427-9216-1da4f7ee7e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.983290 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb4d37-178d-4427-9216-1da4f7ee7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.983567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb4d37-178d-4427-9216-1da4f7ee7e40 admin admin] PATH_INFO: `/v3/projects/8752c061b00543be9c4ab10013d35f8c/users/1f3a16520a33420382ec458db53c6255/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.987731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b541c7-4edd-4d65-afce-ea0bfc0e9dcd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:09.992907 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbd7d7-75c8-427e-b8f5-10e528b2668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:09.993679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbd7d7-75c8-427e-b8f5-10e528b2668a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:09.993956 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbd7d7-75c8-427e-b8f5-10e528b2668a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:09.994226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbd7d7-75c8-427e-b8f5-10e528b2668a admin admin] PATH_INFO: `/v3/projects/c76e12eb7ea5489ca44a96235c1e910d/users/bd2e8fb40e294b31ad92c091680fab20/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:09.998726 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb4d37-178d-4427-9216-1da4f7ee7e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8752c061b00543be9c4ab10013d35f8c, user_id=1f3a16520a33420382ec458db53c6255, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:10.001579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb4d37-178d-4427-9216-1da4f7ee7e40 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:10.010447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbd7d7-75c8-427e-b8f5-10e528b2668a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76e12eb7ea5489ca44a96235c1e910d, user_id=bd2e8fb40e294b31ad92c091680fab20, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:10.013216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbd7d7-75c8-427e-b8f5-10e528b2668a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:10.109318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1347/2698] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:09 2022] PUT /identity/v3/projects/8752c061b00543be9c4ab10013d35f8c/users/1f3a16520a33420382ec458db53c6255/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:10.119567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:10.119854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:10.120127 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:10.143886 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1352/2699] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:09 2022] PUT /identity/v3/projects/c76e12eb7ea5489ca44a96235c1e910d/users/bd2e8fb40e294b31ad92c091680fab20/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:10.148933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] MFA Rules not processed for user `1f3a16520a33420382ec458db53c6255`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:10.149855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:10.150145 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:10.150254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:10.150426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:10.171399 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:10.178925 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:10.219123 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1348/2700] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:10.230463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:10.231807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:10.233115 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1353/2701] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:10.244125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:10.245527 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:10.294392 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69d302-0576-401c-a65f-f31b9570cf95 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:10.304801 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141003f7-7358-4f52-8582-116218e164fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:10.305447 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141003f7-7358-4f52-8582-116218e164fa service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:10.305722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141003f7-7358-4f52-8582-116218e164fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:10.305989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141003f7-7358-4f52-8582-116218e164fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:10.307313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40373e-b12b-462d-8640-36c2c623f588 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:10.311292 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:10.312042 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:10.312392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:10.312675 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:10.315408 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141003f7-7358-4f52-8582-116218e164fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:10.316874 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141003f7-7358-4f52-8582-116218e164fa service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:10.323157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:10.324613 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:10.327870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1349/2702] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:10.336655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1354/2703] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:11.542641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:11.543200 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:11.543406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:11.572769 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] MFA Rules not processed for user `bd2e8fb40e294b31ad92c091680fab20`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:11.574023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:11.587893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:11.642004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1350/2704] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:11.647341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:11.658019 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c526-601f-45d8-aa5c-c3f9b382443d service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:11.664076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:11.665958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:11.666247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:11.666583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:11.667197 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:11.669116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:11.762547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1355/2705] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:11 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:11.766567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:11.777742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7d3a2-fa0c-4d71-aaad-de59861245c3 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:11.834067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:11.834676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:11.834953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:11.835225 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:11.835869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:11.837798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:11.961614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1351/2706] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:11 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:11.965674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:11.977973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6688-d30a-4c4f-a40a-bcc980df96d8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:11.982721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:11.983324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:11.983603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:11.983873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:11.984515 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:11.986136 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:11.999551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1356/2707] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:12.004941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:12.012748 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296db5-c34b-4ba5-9af4-e34e43f3603a admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:12.018872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:12.019532 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:12.019824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:12.020087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] PATH_INFO: `/v3/projects/f863fc538ff74dbba9d377062779e104/users/024082139a4b4dda817bbe669a4f54e3/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:12.037553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f863fc538ff74dbba9d377062779e104, user_id=024082139a4b4dda817bbe669a4f54e3, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:12.040199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:12.150269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1352/2708] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:12 2022] PUT /identity/v3/projects/f863fc538ff74dbba9d377062779e104/users/024082139a4b4dda817bbe669a4f54e3/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:12.154296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:12.164505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbafc2f-56fa-49bb-9cc6-ab30120eb204 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:12.172095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:12.219449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192264e8-12c4-45ba-96de-aa22f7b72ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:12.219977 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192264e8-12c4-45ba-96de-aa22f7b72ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:12.220264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192264e8-12c4-45ba-96de-aa22f7b72ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:12.220540 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192264e8-12c4-45ba-96de-aa22f7b72ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:12.221212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192264e8-12c4-45ba-96de-aa22f7b72ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:12.222798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192264e8-12c4-45ba-96de-aa22f7b72ca7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:12.232280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1357/2709] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:12.236405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:12.246800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fa94-aff9-4342-8ccf-47cd5fab9589 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:12.251308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:12.251943 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:12.252220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:12.252489 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] PATH_INFO: `/v3/projects/f863fc538ff74dbba9d377062779e104/users/024082139a4b4dda817bbe669a4f54e3/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:12.254383 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f863fc538ff74dbba9d377062779e104, user_id=024082139a4b4dda817bbe669a4f54e3, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:12.257681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:12.268119 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1353/2710] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:12 2022] PUT /identity/v3/projects/f863fc538ff74dbba9d377062779e104/users/024082139a4b4dda817bbe669a4f54e3/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:13.491146 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:13.491620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:13.491868 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:13.530700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] MFA Rules not processed for user `024082139a4b4dda817bbe669a4f54e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:13.532044 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:13.609573 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1358/2711] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:13.622822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:13.688802 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b3647-9a11-4e6d-82f6-3fac81220542 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:13.693508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:13.694156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:13.694260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:13.694503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:13.715216 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:13.750882 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:13.752321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:13.762960 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1354/2712] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:16.412673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:16.422132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607ba18-88e2-434b-a710-a2456ee9520d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:16.425722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:16.426311 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:16.426587 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:16.427383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:16.436729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:16.438232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:16.448518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1359/2713] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:23.788634 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=17 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:26.444471 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=64 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:34.492681 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:34.517952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:34.519330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:34.619924 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76c0f6-468b-45bc-a73e-fed9b96cd175 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:34.625985 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:34.626553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:34.626967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:34.627278 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] PATH_INFO: `/v3/users/1f3a16520a33420382ec458db53c6255` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:34.628650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3a16520a33420382ec458db53c6255)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:34.631163 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:34.708390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0eaaf-2ce9-47a0-bf33-699c9b0e2c22 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:34.714417 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:34.715018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:34.715294 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:34.715562 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:34.717318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:34.729233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:34.730790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8eb034-80f1-47d7-88bb-73590e19b066 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:34.741695 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1360/2714] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:34.881953 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3a16520a33420382ec458db53c6255'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:34.988249 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3a16520a33420382ec458db53c6255'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.000823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3a16520a33420382ec458db53c6255'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.000823 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3a16520a33420382ec458db53c6255'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.009825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3a16520a33420382ec458db53c6255'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.015822 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1355/2715] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:34 2022] DELETE /identity/v3/users/1f3a16520a33420382ec458db53c6255 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:35.020066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:35.020418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:35.020761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:35.067074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:35.069133 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:35.133100 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1361/2716] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:35.142523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:35.143939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:35.187879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:35.223388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b192f8-574a-49d4-8e63-073b00a1de7b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:35.228218 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:35.228870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:35.229184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:35.229460 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:35.238610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:35.240614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:35.250396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1356/2717] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:35.445188 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:35.446787 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:35.455489 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:35.460272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1aaf-1f72-469e-b4e8-64d2595bc3d8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:35.465178 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:35.465800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:35.466081 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:35.466354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] PATH_INFO: `/v3/users/bd2e8fb40e294b31ad92c091680fab20` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:35.467492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2e8fb40e294b31ad92c091680fab20)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:35.469445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:35.639032 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2e8fb40e294b31ad92c091680fab20'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.654655 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2e8fb40e294b31ad92c091680fab20'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.731574 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2e8fb40e294b31ad92c091680fab20'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.732227 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2e8fb40e294b31ad92c091680fab20'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.745740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2e8fb40e294b31ad92c091680fab20'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:35.752203 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1362/2718] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:35 2022] DELETE /identity/v3/users/bd2e8fb40e294b31ad92c091680fab20 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:35.889164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:35.889164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:35.905021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052fb27-85db-4c37-9442-b8a8c9f8bc57 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:35.987802 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:35.988348 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:35.988650 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:35.988920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] PATH_INFO: `/v3/projects/8752c061b00543be9c4ab10013d35f8c` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:35.990923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8752c061b00543be9c4ab10013d35f8c)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:35.992817 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.008355 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.010101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:36.024583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa2a7b-05fb-4a9b-9ea3-d29212de466e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.029264 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.030070 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.030351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.030621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] PATH_INFO: `/v3/projects/c76e12eb7ea5489ca44a96235c1e910d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.031741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76e12eb7ea5489ca44a96235c1e910d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.033704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.132345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8752c061b00543be9c4ab10013d35f8c', 'request_id': 'req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.137938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8752c061b00543be9c4ab10013d35f8c', 'request_id': 'req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.144042 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8752c061b00543be9c4ab10013d35f8c', 'request_id': 'req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.166361 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76e12eb7ea5489ca44a96235c1e910d', 'request_id': 'req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.171774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76e12eb7ea5489ca44a96235c1e910d', 'request_id': 'req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.178187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76e12eb7ea5489ca44a96235c1e910d', 'request_id': 'req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.233344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] The token cache is being invalidate because project 8752c061b00543be9c4ab10013d35f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:36.233619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.234243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-df0c1f1c-5e71-43c6-a9ac-95ad0e0c08a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.235723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1357/2719] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:35 2022] DELETE /identity/v3/projects/8752c061b00543be9c4ab10013d35f8c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:36.240472 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95dbd-d949-4edf-97aa-4acede9d0ac5 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.240691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95dbd-d949-4edf-97aa-4acede9d0ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.240962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95dbd-d949-4edf-97aa-4acede9d0ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.288555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-1ee95dbd-d949-4edf-97aa-4acede9d0ac5 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:36.289806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1ee95dbd-d949-4edf-97aa-4acede9d0ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:36.296827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] The token cache is being invalidate because project c76e12eb7ea5489ca44a96235c1e910d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:36.301593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.301593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.301593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1363/2720] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:36 2022] DELETE /identity/v3/projects/c76e12eb7ea5489ca44a96235c1e910d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:36.303498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.304855 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:36.311963 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:36.351697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1358/2721] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:36.359676 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.359967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.360245 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.374103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4111-4ac9-4791-9f56-7c3cfdf9ced7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.379120 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.379753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.380035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.380280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:36.380391 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] PATH_INFO: `/v3/users/024082139a4b4dda817bbe669a4f54e3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.381722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024082139a4b4dda817bbe669a4f54e3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.382310 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:36.383656 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.385232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:36.390265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1359/2722] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:36.394356 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.404064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31bcdf-9c79-4e8c-9214-a7bf12712fe8 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.409434 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.410045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.410503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.411026 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.412461 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:36.412807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.414547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.425790 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1360/2723] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:33:36.430717 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.440105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f9a6-cdb8-4d7f-9cd5-1f75ab3b6b15 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.445457 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.446008 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.446284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.446558 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.447142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.448948 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.531179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1361/2724] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:36 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:36.536006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.596153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63107e26-041a-450a-ab82-58e3de8980a0 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.601795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.602373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.602649 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.602921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.603526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.605678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.649364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024082139a4b4dda817bbe669a4f54e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.656413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024082139a4b4dda817bbe669a4f54e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.743791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1362/2725] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:36 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:36.748062 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.757632 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed126349-4bdf-4659-ad27-d16a79378ad2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.762321 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.762892 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.763175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.763450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.764141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.765775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.766156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024082139a4b4dda817bbe669a4f54e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.766881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024082139a4b4dda817bbe669a4f54e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.775849 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1363/2726] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:36.779738 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.779870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024082139a4b4dda817bbe669a4f54e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:36.781531 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:36.786022 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1364/2727] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:36 2022] DELETE /identity/v3/users/024082139a4b4dda817bbe669a4f54e3 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:33:36.795100 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3ec53-6cc5-43d2-8192-e79d94454e93 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.853338 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f10fd-d6a4-4d0e-b34a-79db31fac967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.854088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f10fd-d6a4-4d0e-b34a-79db31fac967 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.854371 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f10fd-d6a4-4d0e-b34a-79db31fac967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.854781 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f10fd-d6a4-4d0e-b34a-79db31fac967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.855438 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f10fd-d6a4-4d0e-b34a-79db31fac967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.857061 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f10fd-d6a4-4d0e-b34a-79db31fac967 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:36.862233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:36.868904 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1364/2728] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:36.875975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:36.886363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3038b0e-6530-4d21-b04e-2da73403d7f8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:36.892082 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:36.892741 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:36.893072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:36.893356 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] PATH_INFO: `/v3/projects/b886505ac3df49769d17d31b8d3ab7e1/users/ff83c25b3c8a4dd1920d08368d469be9/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:36.909832 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b886505ac3df49769d17d31b8d3ab7e1, user_id=ff83c25b3c8a4dd1920d08368d469be9, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:36.913216 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.016583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1365/2729] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:36 2022] PUT /identity/v3/projects/b886505ac3df49769d17d31b8d3ab7e1/users/ff83c25b3c8a4dd1920d08368d469be9/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:37.028562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.029104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.029219 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.041287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.043230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:37.060799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] MFA Rules not processed for user `ff83c25b3c8a4dd1920d08368d469be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:37.062179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:37.125775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13ee67-f2d8-433f-b05f-38ef512557b8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.130732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.131401 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1365/2730] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:37.134874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.134874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.134874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] PATH_INFO: `/v3/projects/f863fc538ff74dbba9d377062779e104` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.134874 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f863fc538ff74dbba9d377062779e104)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.138059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.142069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.143451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:37.200297 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48f822-f44e-44e5-aaba-25cea3e7976d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.203966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a3e32-b86e-46f5-bc6a-f5d5116fa1a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.204561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a3e32-b86e-46f5-bc6a-f5d5116fa1a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.204843 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a3e32-b86e-46f5-bc6a-f5d5116fa1a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.205147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a3e32-b86e-46f5-bc6a-f5d5116fa1a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.214090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a3e32-b86e-46f5-bc6a-f5d5116fa1a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.215507 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a3e32-b86e-46f5-bc6a-f5d5116fa1a4 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.224679 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1366/2731] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:33:37.246248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f863fc538ff74dbba9d377062779e104', 'request_id': 'req-4128e3bc-2fcc-4677-a398-bd903f2c0885'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:37.250885 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:37.252175 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f863fc538ff74dbba9d377062779e104', 'request_id': 'req-4128e3bc-2fcc-4677-a398-bd903f2c0885'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:37.259279 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f863fc538ff74dbba9d377062779e104', 'request_id': 'req-4128e3bc-2fcc-4677-a398-bd903f2c0885'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:37.337460 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] The token cache is being invalidate because project f863fc538ff74dbba9d377062779e104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:37.337762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:37.338405 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4128e3bc-2fcc-4677-a398-bd903f2c0885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:37.339927 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1366/2732] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:37 2022] DELETE /identity/v3/projects/f863fc538ff74dbba9d377062779e104 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:37.345162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.345709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.345941 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.365841 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:37.367043 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:37.425706 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1367/2733] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:37.433916 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.436568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.436568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.454303 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:37.456462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:37.465432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1367/2734] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:37.469529 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.479319 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d75206-250f-4c95-86bd-22690bad5d62 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.483860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.484496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.486665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.486665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.486665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:37.486665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.487555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.498972 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1368/2735] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:33:37.503354 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.513218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0469aa7-9893-4081-8b83-0b5439be4aaf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.517938 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.518529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.518805 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.519074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.519719 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.521752 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.613009 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1368/2736] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:37 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:33:37.617835 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.627739 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16353c1-7bfd-419f-8f8f-88bba0477416 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.644668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=100 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:37.685395 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.686010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.690071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.690071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.690071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.690071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.796375 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1369/2737] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:37 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:37.799097 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.810486 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befa8c1-c868-4bf5-afce-415b3ccc2817 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.816236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.816236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.816236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.816550 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.817066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.819630 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.829810 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1369/2738] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:37.834254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.844252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc9fb7-7994-4a89-b766-711a3bcc645d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.848855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f923fc-59ef-442c-803d-b959d9f974e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.849945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f923fc-59ef-442c-803d-b959d9f974e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.849945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f923fc-59ef-442c-803d-b959d9f974e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.850149 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f923fc-59ef-442c-803d-b959d9f974e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.850770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f923fc-59ef-442c-803d-b959d9f974e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.852343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f923fc-59ef-442c-803d-b959d9f974e5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.862716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1370/2739] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:37.867327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:37.877674 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca82076-fb98-4927-9ee3-c5c3cf4847af admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:37.883890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:37.885571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.885571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.885571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] PATH_INFO: `/v3/projects/ded676e84f4145d5912a3878ef9af132/users/9675d83e26b647489e4b6195bb814aab/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.900287 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded676e84f4145d5912a3878ef9af132, user_id=9675d83e26b647489e4b6195bb814aab, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:37.903105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:37.954440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1370/2740] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:37 2022] PUT /identity/v3/projects/ded676e84f4145d5912a3878ef9af132/users/9675d83e26b647489e4b6195bb814aab/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:37.959428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837fdbe-2834-4367-ada6-12f4d18bbd58 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:37.959719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837fdbe-2834-4367-ada6-12f4d18bbd58 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:37.959991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837fdbe-2834-4367-ada6-12f4d18bbd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:37.978318 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e837fdbe-2834-4367-ada6-12f4d18bbd58 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:37.979519 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e837fdbe-2834-4367-ada6-12f4d18bbd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:38.041990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1371/2741] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:38.051558 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:38.054185 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:38.111569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fce001-4341-4831-ba80-7055d7e548ca admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:38.115231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:38.115830 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:38.118775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:38.118775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:38.125362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:38.126769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:38.136451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1371/2742] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:43.454096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:43.454619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:43.454619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:43.483559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] MFA Rules not processed for user `9675d83e26b647489e4b6195bb814aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:43.484794 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:43.566417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1372/2743] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:43.571493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:43.581288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46bf2-80f1-4159-8048-fa6c25a81c7e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:43.586244 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:43.586948 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:43.587301 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:43.587646 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:43.588353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:43.590383 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:43.716721 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1372/2744] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:43 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:43.720762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:43.733090 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df0ab-40be-4463-88ca-719cb1961b4f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:43.760699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:43.794281 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:43.794909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:43.795197 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:43.795475 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:43.796128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:43.798302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:43.925685 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1373/2745] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:43 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:43.929645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:43.945168 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd89c-a543-4207-9643-a713e495c416 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:43.951020 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:43.951835 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:43.952274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:43.952629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:43.953339 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:43.956312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:43.971892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1373/2746] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:33:43.976925 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:43.988096 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83062ab8-7a32-49ab-9bec-c3eac338a118 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:43.992911 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:43.993612 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:43.993898 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:43.994171 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] PATH_INFO: `/v3/projects/39593d796c77478b990a9d5bc90e176d/users/8076a76979f048359da245ac01300429/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:44.011169 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39593d796c77478b990a9d5bc90e176d, user_id=8076a76979f048359da245ac01300429, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:44.014012 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:44.109205 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1374/2747] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:43 2022] PUT /identity/v3/projects/39593d796c77478b990a9d5bc90e176d/users/8076a76979f048359da245ac01300429/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:44.113288 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:44.124790 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a373db-7c93-4aa6-8a03-e3f2126bfeaf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:44.174557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:44.184462 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96ec6c-8884-4f41-802b-60bc14e09434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:44.186890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96ec6c-8884-4f41-802b-60bc14e09434 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:44.186890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96ec6c-8884-4f41-802b-60bc14e09434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:44.186890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96ec6c-8884-4f41-802b-60bc14e09434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:44.186890 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96ec6c-8884-4f41-802b-60bc14e09434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:44.187876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96ec6c-8884-4f41-802b-60bc14e09434 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:44.198568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1374/2748] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:33:44.202910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:44.214125 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d810e-1411-46b8-9735-562e18bb9778 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:44.218959 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:44.219600 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:44.219883 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:44.220174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] PATH_INFO: `/v3/projects/39593d796c77478b990a9d5bc90e176d/users/8076a76979f048359da245ac01300429/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:44.222138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39593d796c77478b990a9d5bc90e176d, user_id=8076a76979f048359da245ac01300429, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:44.225211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:44.239004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1375/2749] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:33:44 2022] PUT /identity/v3/projects/39593d796c77478b990a9d5bc90e176d/users/8076a76979f048359da245ac01300429/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:49.806401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:49.807003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:49.807003 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:49.836568 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] MFA Rules not processed for user `8076a76979f048359da245ac01300429`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:49.837823 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:49.920931 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1375/2750] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:49.937246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:49.939069 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:50.065597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c164e8f-e2d9-422d-97ba-bae082212b61 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:50.072644 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202eb4f-a80a-48c7-a1ce-58cc3e6033e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:50.073303 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202eb4f-a80a-48c7-a1ce-58cc3e6033e0 service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:50.073584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202eb4f-a80a-48c7-a1ce-58cc3e6033e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:50.073907 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202eb4f-a80a-48c7-a1ce-58cc3e6033e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:50.126451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:50.135156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202eb4f-a80a-48c7-a1ce-58cc3e6033e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:50.136795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202eb4f-a80a-48c7-a1ce-58cc3e6033e0 service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:50.147081 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1376/2751] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:58.283157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:58.284833 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:58.352382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51653f2c-fd00-448c-b6a9-d6ae2cd42b6c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:58.356892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:58.357593 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:58.357899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:58.358172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] PATH_INFO: `/v3/users/ff83c25b3c8a4dd1920d08368d469be9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:58.359278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff83c25b3c8a4dd1920d08368d469be9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:58.361206 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:58.522407 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff83c25b3c8a4dd1920d08368d469be9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:58.530016 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff83c25b3c8a4dd1920d08368d469be9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:58.580461 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff83c25b3c8a4dd1920d08368d469be9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:58.581105 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff83c25b3c8a4dd1920d08368d469be9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:58.591909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:58.597528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff83c25b3c8a4dd1920d08368d469be9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:58.604374 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1376/2752] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:33:58 2022] DELETE /identity/v3/users/ff83c25b3c8a4dd1920d08368d469be9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:58.608915 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:58.619075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:58.619075 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:58.627863 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:58.629095 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:58.692562 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1377/2753] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:33:58.702826 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:58.704286 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:58.765730 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a0ca3-efdc-4615-842a-63bbef9499af admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:58.769609 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cad42b-bd7f-4d48-825b-902f30405976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:58.770121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad42b-bd7f-4d48-825b-902f30405976 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:58.770403 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad42b-bd7f-4d48-825b-902f30405976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:58.770683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad42b-bd7f-4d48-825b-902f30405976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:58.785066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cad42b-bd7f-4d48-825b-902f30405976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:58.785066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cad42b-bd7f-4d48-825b-902f30405976 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:58.792074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1377/2754] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:59.223991 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:59.226109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:59.238150 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaa9dd-1ca2-45fd-a01c-a00c078f7451 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:59.244349 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:59.245015 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:59.245289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:59.246645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] PATH_INFO: `/v3/projects/b886505ac3df49769d17d31b8d3ab7e1` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:59.246788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b886505ac3df49769d17d31b8d3ab7e1)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:59.248598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:59.393723 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b886505ac3df49769d17d31b8d3ab7e1', 'request_id': 'req-cadefd74-09c9-4694-bb6d-c005d5f336da'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:59.400581 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b886505ac3df49769d17d31b8d3ab7e1', 'request_id': 'req-cadefd74-09c9-4694-bb6d-c005d5f336da'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:59.408229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b886505ac3df49769d17d31b8d3ab7e1', 'request_id': 'req-cadefd74-09c9-4694-bb6d-c005d5f336da'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:59.542564 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] The token cache is being invalidate because project b886505ac3df49769d17d31b8d3ab7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:33:59.542862 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:59.543488 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cadefd74-09c9-4694-bb6d-c005d5f336da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:33:59.545025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1378/2755] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:33:59 2022] DELETE /identity/v3/projects/b886505ac3df49769d17d31b8d3ab7e1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:33:59.560438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:59.560438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:59.560438 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:59.571516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:59.572745 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:59.658331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1378/2756] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:59.666910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:59.667427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:59.667427 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:59.688818 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:33:59.690063 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:33:59.698678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1379/2757] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:33:59.703144 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:59.714080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f447c-7594-4d68-93dd-639b190a8759 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:59.719432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:59.720043 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:59.720431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:59.720640 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:59.721327 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:33:59.721636 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:59.723226 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:59.739151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1379/2758] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:33:59.740374 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:59.757411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79482151-b07f-427a-b079-09bfae5fddd1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:59.762289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:59.762935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:59.766416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:59.766416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:59.766416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:59.766416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:33:59.845585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1380/2759] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:33:59 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:33:59.849649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:33:59.860412 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371213-6e66-4aa9-b8b5-50759d521be0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:33:59.922894 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:33:59.923516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:33:59.923800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:33:59.924078 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:33:59.924702 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:33:59.926648 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:00.015447 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1380/2760] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:33:59 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:00.019775 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:00.031232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8e253-b57f-4c8e-bead-b631e140ddf5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:00.036458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:00.037091 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:00.037373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:00.037641 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:00.039288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:00.041208 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:00.051768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1381/2761] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:00.056521 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:00.066834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b572-7e8d-4983-8493-2bf7d198734d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:00.071414 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbed381-89b7-4536-9beb-d1c3371f42ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:00.072056 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbed381-89b7-4536-9beb-d1c3371f42ad admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:00.072283 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbed381-89b7-4536-9beb-d1c3371f42ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:00.072556 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbed381-89b7-4536-9beb-d1c3371f42ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:00.073233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbed381-89b7-4536-9beb-d1c3371f42ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:00.074961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbed381-89b7-4536-9beb-d1c3371f42ad admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:00.080484 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:00.087132 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1381/2762] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:00.091237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:00.103344 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d704860-3704-42e3-aadd-f8de33434448 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:00.108204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:00.108855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:00.109164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:00.109436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] PATH_INFO: `/v3/projects/1596bc6318b24822beda3dc89834c7b2/users/e4d96e16d368455aa92e1cd62219f106/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:00.124283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1596bc6318b24822beda3dc89834c7b2, user_id=e4d96e16d368455aa92e1cd62219f106, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:00.127047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:00.211986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1382/2763] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:00 2022] PUT /identity/v3/projects/1596bc6318b24822beda3dc89834c7b2/users/e4d96e16d368455aa92e1cd62219f106/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:00.217313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:00.217622 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:00.217903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:00.238189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:00.241160 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:00.302607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1382/2764] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:00.312258 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:00.313970 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:00.328364 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:00.370744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7325b2a-5a7b-4b1e-8c74-af305ae04e3d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:00.374449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:00.375077 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:00.375357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:00.375627 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:00.385415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:00.386894 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:00.398058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1383/2765] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:00.796762 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:00.807443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c58f3-22fe-4d42-9126-9d1470801394 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:00.811031 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:00.811629 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:00.811904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:00.812173 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:00.814123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:00.826975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:00.828426 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:00.840324 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1383/2766] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:05.347829 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:05.349840 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:05.368708 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c29c9-87e2-425a-b1e9-96ceb1ca3a3b service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:05.375694 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:05.376854 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:05.377410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:05.378080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] PATH_INFO: `/v3/users/1945fed03f73472db818c4e5ca015eb7` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:05.379523 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1945fed03f73472db818c4e5ca015eb7)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:05.382742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:05.573332 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1945fed03f73472db818c4e5ca015eb7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.626889 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1945fed03f73472db818c4e5ca015eb7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.639078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1945fed03f73472db818c4e5ca015eb7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.639078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1945fed03f73472db818c4e5ca015eb7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.662328 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4f588d0-355a-42a2-95bd-104c3660f05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1945fed03f73472db818c4e5ca015eb7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.673072 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1384/2767] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:05 2022] DELETE /identity/v3/users/1945fed03f73472db818c4e5ca015eb7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:05.939098 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:05.951814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04713f6-9af1-462b-897f-5fe935bef251 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.035478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.036278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.036563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.038516 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] PATH_INFO: `/v3/projects/a60a5752fcba46359ef5275f506c933a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.039246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60a5752fcba46359ef5275f506c933a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.041265 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.171372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60a5752fcba46359ef5275f506c933a', 'request_id': 'req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.177920 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60a5752fcba46359ef5275f506c933a', 'request_id': 'req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.188047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60a5752fcba46359ef5275f506c933a', 'request_id': 'req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.263566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] The token cache is being invalidate because project a60a5752fcba46359ef5275f506c933a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:06.264047 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.264751 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c30c0f43-3f8e-43ec-8e3a-45f294d5b932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.272663 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1384/2768] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:05 2022] DELETE /identity/v3/projects/a60a5752fcba46359ef5275f506c933a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:06.274678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.276900 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.277358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.311807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:06.313454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:06.388642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1385/2769] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:06.397861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.398861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.399545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.425390 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:06.426875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:06.436621 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1385/2770] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:06.440895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.452437 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3db2-956a-49eb-a545-e5d4c7b0dab4 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.457618 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.458246 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.458530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.458806 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.459579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:34:06.459989 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.461752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fda701-57ac-4600-aefb-f0a85c8a33e7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.474343 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1386/2771] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:34:06.479946 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.492546 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed6960-8a4a-488a-80ce-b6d812796a3b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.504649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.505314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.505594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.505861 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.506561 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.508525 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.519384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:06.578983 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.579274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.594772 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.617813 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:06.617975 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1386/2772] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:06.619404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] MFA Rules not processed for user `e4d96e16d368455aa92e1cd62219f106`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:06.619641 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.620824 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:06.692380 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1387/2773] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:06.694811 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc598-89fc-476f-b2aa-86b914fc71c5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.697147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.698608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:06.700514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.701182 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.703704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.703704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.703704 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.705172 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.712229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79764c0-6fbc-4ef0-a07b-89593f4a1848 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.717593 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.718406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.718958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.719430 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.720538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.722869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.806548 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1387/2774] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:06.810921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.819047 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1388/2775] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:34:06.823175 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.827531 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020f44-4201-4a9b-9c31-b1292fd8fada admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.890449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d36f-421d-47b2-a167-84a892648f7d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.893539 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.894156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.894435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.894715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.895364 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.895503 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.896110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.896394 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.896697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.897545 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.898111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.900045 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.907870 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1388/2776] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:34:06.912202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.922563 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea7f1-8ad1-41b7-95bb-e4625441266c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.927233 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.927819 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.928130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.928406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.929074 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.930694 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.941107 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1389/2777] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:34:06.945444 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:06.955093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9694f-06f1-48ee-8d6e-a795c7325694 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:06.960514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e04db3-44a9-4ec7-b388-a0cdd7e93839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:06.961207 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e04db3-44a9-4ec7-b388-a0cdd7e93839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:06.961492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e04db3-44a9-4ec7-b388-a0cdd7e93839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:06.961766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e04db3-44a9-4ec7-b388-a0cdd7e93839 admin admin] PATH_INFO: `/v3/projects/e3abef1896be4be48de68e306f717290/users/f9500563b1824e60be6dc86f48fb529d/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:06.976651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e04db3-44a9-4ec7-b388-a0cdd7e93839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3abef1896be4be48de68e306f717290, user_id=f9500563b1824e60be6dc86f48fb529d, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:06.979320 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e04db3-44a9-4ec7-b388-a0cdd7e93839 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.988405 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1389/2778] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:06 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:34:06.992758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.002456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d56e81-d43a-4bfb-a82a-c52fa06ddfa2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.007139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.007719 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.008002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.008279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.008921 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.010551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.020273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1390/2779] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:34:07.024720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.034642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeea75e-f9f0-41c0-bb72-cfa62d04b729 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.039962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.040617 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.040920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.041231 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] PATH_INFO: `/v3/projects/5817f311a0c0410db02c9da5e334e67c/users/cae338755bbb49b8911bfe2ef9d0ce09/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.057180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1390/2780] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:06 2022] PUT /identity/v3/projects/e3abef1896be4be48de68e306f717290/users/f9500563b1824e60be6dc86f48fb529d/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:34:07.066935 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5817f311a0c0410db02c9da5e334e67c, user_id=cae338755bbb49b8911bfe2ef9d0ce09, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.068579 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.068909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.069173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.069393 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.096738 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] MFA Rules not processed for user `f9500563b1824e60be6dc86f48fb529d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:07.097973 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:07.127099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:07.142372 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1391/2781] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:07 2022] PUT /identity/v3/projects/5817f311a0c0410db02c9da5e334e67c/users/cae338755bbb49b8911bfe2ef9d0ce09/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:34:07.146570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.164596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1391/2782] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:07 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:07.169211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.188579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:07.213819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6ff7d-dcb9-4539-b6ee-8db5d0e5ea00 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.218548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.219226 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.220060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.220060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.220454 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.222104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.233330 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1392/2783] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:07.235547 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d23c66-8d0a-4de6-b5ba-02ff911a1246 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.236967 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.243771 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.244766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.244766 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.244963 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.245705 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.247657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.248153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6c71d-2ffb-4b66-9221-a91eb7d2f9e2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.253196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.254041 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.254415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.254786 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] PATH_INFO: `/v3/projects/5817f311a0c0410db02c9da5e334e67c/users/cae338755bbb49b8911bfe2ef9d0ce09/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.257538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5817f311a0c0410db02c9da5e334e67c, user_id=cae338755bbb49b8911bfe2ef9d0ce09, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.260656 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.274055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1393/2784] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:07 2022] PUT /identity/v3/projects/5817f311a0c0410db02c9da5e334e67c/users/cae338755bbb49b8911bfe2ef9d0ce09/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:34:07.324529 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1392/2785] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:07 2022] POST /identity/v3/projects => generated 405 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:34:07.325551 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.391633 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787aa7c-b744-4bac-b895-017e8d3b97d1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.469053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.469750 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.470103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.470446 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.471154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.473311 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.565342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1394/2786] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:07 2022] POST /identity/v3/users => generated 309 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:07.570059 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.592337 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ac47b-6cb2-4df8-82fa-9f92528c895c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.599638 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.606384 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.606682 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.606966 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.607737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.610317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.626599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1393/2787] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:07.633838 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.644514 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbac868-26a6-4848-abd1-c4f0f9ebd4dd admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.656466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.657955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.658484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.659268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] PATH_INFO: `/v3/projects/318b5d730d9648a9a48781940b2672ae/users/f761f148967a4c70a3499bb03692c8c9/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.675944 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318b5d730d9648a9a48781940b2672ae, user_id=f761f148967a4c70a3499bb03692c8c9, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.679988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.752456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1395/2788] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:07 2022] PUT /identity/v3/projects/318b5d730d9648a9a48781940b2672ae/users/f761f148967a4c70a3499bb03692c8c9/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:07.753732 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.767440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a0e98-cabd-479b-98b5-27065cf87776 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.844487 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ad9ce-a3bc-45a9-8198-dc8ca72517f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.845722 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ad9ce-a3bc-45a9-8198-dc8ca72517f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.846495 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ad9ce-a3bc-45a9-8198-dc8ca72517f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.847633 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ad9ce-a3bc-45a9-8198-dc8ca72517f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.848411 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ad9ce-a3bc-45a9-8198-dc8ca72517f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.850369 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ad9ce-a3bc-45a9-8198-dc8ca72517f3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.863413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1394/2789] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:07.869692 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:07.880541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4cee8-2354-4225-8126-fde178185ee8 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:07.886058 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:07.886699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.886979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.887247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] PATH_INFO: `/v3/projects/318b5d730d9648a9a48781940b2672ae/users/f761f148967a4c70a3499bb03692c8c9/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.889173 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] RBAC: Authorizing `identity:create_grant(project_id=318b5d730d9648a9a48781940b2672ae, user_id=f761f148967a4c70a3499bb03692c8c9, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:07.891940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:07.907825 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1396/2790] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:07 2022] PUT /identity/v3/projects/318b5d730d9648a9a48781940b2672ae/users/f761f148967a4c70a3499bb03692c8c9/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:07.916228 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:07.916526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:07.916804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:07.974001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] MFA Rules not processed for user `f761f148967a4c70a3499bb03692c8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:07.977904 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:08.017720 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:08.064312 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1395/2791] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:08.080202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:08.086463 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:08.114182 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:08.181697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086732a-2ea9-426d-b59b-9b84be73631c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:08.185678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eccbc2-b1fb-4c54-ae0c-f315be5a0d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:08.186433 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eccbc2-b1fb-4c54-ae0c-f315be5a0d13 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:08.186718 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eccbc2-b1fb-4c54-ae0c-f315be5a0d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:08.186990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eccbc2-b1fb-4c54-ae0c-f315be5a0d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:08.197222 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eccbc2-b1fb-4c54-ae0c-f315be5a0d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:08.198657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eccbc2-b1fb-4c54-ae0c-f315be5a0d13 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:08.208138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1397/2792] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:08 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:08.380351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:08.449400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aea1e-0a65-47c1-8947-7676a0c093a2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:08.457659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:08.457659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:08.457659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:08.457659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] PATH_INFO: `/v3/users/f9500563b1824e60be6dc86f48fb529d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:08.458025 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9500563b1824e60be6dc86f48fb529d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:08.458269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:08.675821 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9500563b1824e60be6dc86f48fb529d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:08.696184 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9500563b1824e60be6dc86f48fb529d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:08.789125 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9500563b1824e60be6dc86f48fb529d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:08.790435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9500563b1824e60be6dc86f48fb529d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:08.807439 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9500563b1824e60be6dc86f48fb529d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:08.815217 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1396/2793] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:08 2022] DELETE /identity/v3/users/f9500563b1824e60be6dc86f48fb529d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:08.820888 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:08.826640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:08.826640 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:08.843124 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:08.844370 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:08.914014 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1398/2794] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:08.924926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:08.926512 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:08.979246 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:08.991660 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7a90-b32f-4a7b-94a7-3ec35e426c4a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:08.996952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:08.997617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:08.997881 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:08.998158 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:09.008066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:09.009739 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:09.019765 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1397/2795] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:09.642456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:09.644220 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:09.663674 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796eda4-f5be-4272-8b21-c78ddc9af91e None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:09.669487 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:09.671795 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:09.672158 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:09.672509 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] PATH_INFO: `/v3/projects/e3abef1896be4be48de68e306f717290` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:09.674189 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3abef1896be4be48de68e306f717290)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:09.676283 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:09.806342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3abef1896be4be48de68e306f717290', 'request_id': 'req-ad6b8270-2e2d-403e-83d5-03de9b00ae24'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:09.813104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3abef1896be4be48de68e306f717290', 'request_id': 'req-ad6b8270-2e2d-403e-83d5-03de9b00ae24'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:09.828242 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3abef1896be4be48de68e306f717290', 'request_id': 'req-ad6b8270-2e2d-403e-83d5-03de9b00ae24'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:09.900668 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] The token cache is being invalidate because project e3abef1896be4be48de68e306f717290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:09.900968 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:09.905937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-ad6b8270-2e2d-403e-83d5-03de9b00ae24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:09.909691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1399/2796] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:09 2022] DELETE /identity/v3/projects/e3abef1896be4be48de68e306f717290 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:09.914000 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:09.939278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5a189-112a-420c-bace-afd87b050590 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:10.040449 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:10.041483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:10.041932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:10.042215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] PATH_INFO: `/v3/users/f761f148967a4c70a3499bb03692c8c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:10.045575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f761f148967a4c70a3499bb03692c8c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:10.047566 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:10.323987 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f761f148967a4c70a3499bb03692c8c9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:10.332365 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f761f148967a4c70a3499bb03692c8c9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:10.423171 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f761f148967a4c70a3499bb03692c8c9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:10.423928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f761f148967a4c70a3499bb03692c8c9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:10.441598 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f761f148967a4c70a3499bb03692c8c9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:10.449801 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1398/2797] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:09 2022] DELETE /identity/v3/users/f761f148967a4c70a3499bb03692c8c9 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:10.810567 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58044165-353a-4246-8e8e-57debc4c7680 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:10.810881 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58044165-353a-4246-8e8e-57debc4c7680 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:10.811162 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58044165-353a-4246-8e8e-57debc4c7680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:10.844421 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-58044165-353a-4246-8e8e-57debc4c7680 None None] MFA Rules not processed for user `9675d83e26b647489e4b6195bb814aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:10.846645 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-58044165-353a-4246-8e8e-57debc4c7680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:10.875233 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:10.935815 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1400/2798] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:11.017373 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:11.018954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:11.152366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4657e-ce19-4f2d-bbe2-f7b4237e4635 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:11.158157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:11.160218 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:11.161099 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:11.161534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] PATH_INFO: `/v3/projects/318b5d730d9648a9a48781940b2672ae` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:11.164445 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] RBAC: Authorizing `identity:delete_project(project_id=318b5d730d9648a9a48781940b2672ae)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:11.166736 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:11.177549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:11.291116 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318b5d730d9648a9a48781940b2672ae', 'request_id': 'req-ed26b287-72e2-41f5-9389-c6377608955a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:11.313958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318b5d730d9648a9a48781940b2672ae', 'request_id': 'req-ed26b287-72e2-41f5-9389-c6377608955a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:11.321483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318b5d730d9648a9a48781940b2672ae', 'request_id': 'req-ed26b287-72e2-41f5-9389-c6377608955a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:11.389343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] The token cache is being invalidate because project 318b5d730d9648a9a48781940b2672ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:11.389631 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:11.390620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ed26b287-72e2-41f5-9389-c6377608955a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:11.397274 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1399/2799] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:11 2022] DELETE /identity/v3/projects/318b5d730d9648a9a48781940b2672ae => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:11.399969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:11.400269 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:11.400548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:11.431752 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:11.435500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:11.520030 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1401/2800] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:11.528151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:11.528437 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:11.528711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:11.565090 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:11.566382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:11.586899 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1400/2801] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:11.591992 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:11.608455 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b77a6-ea67-450a-8fb2-376225058c7f None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:11.620247 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:11.622031 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:11.623834 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:11.625802 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:11.627912 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:34:11.628844 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:11.636597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:11.682304 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1402/2802] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:34:11.682413 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:11.697404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ec7f8-8f5c-4d7c-89b4-c901c0c2a0fa None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:11.704236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:11.705297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:11.710785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:11.710785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:11.710785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:11.710785 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:11.811214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1401/2803] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:11 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:11.812774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:11.824614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e85be-dc64-4688-9e86-80e7e54fd6d1 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:11.916250 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:11.920154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:11.920154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:11.920154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:11.920154 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:11.923161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:12.046317 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1403/2804] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:11 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:34:12.050420 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:12.062839 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edede3-bb15-47c9-b2ba-a09f6d58bf0f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:12.069435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:12.070064 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.070350 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.070626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.071524 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:12.073278 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:12.086256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1402/2805] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:12.090360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:12.105360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d2ec-6155-49d6-a3b5-75bdaa8a9176 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:12.112879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0c26d-8acd-4a49-8e5f-c768fe80dcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:12.114619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0c26d-8acd-4a49-8e5f-c768fe80dcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.115213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0c26d-8acd-4a49-8e5f-c768fe80dcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.116209 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0c26d-8acd-4a49-8e5f-c768fe80dcf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.117260 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0c26d-8acd-4a49-8e5f-c768fe80dcf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:12.119103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0c26d-8acd-4a49-8e5f-c768fe80dcf1 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:12.142050 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1404/2806] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:12.146381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:12.158784 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac3da1-b960-4ae0-a737-4c651b41a6dc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:12.165225 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:12.166112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.166793 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.167224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] PATH_INFO: `/v3/projects/07d13793ff6d4f14b436bbced1aa4de2/users/f87045df238146108914b050fa2f3378/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.189135 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d13793ff6d4f14b436bbced1aa4de2, user_id=f87045df238146108914b050fa2f3378, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:12.193957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:12.276029 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1403/2807] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:12 2022] PUT /identity/v3/projects/07d13793ff6d4f14b436bbced1aa4de2/users/f87045df238146108914b050fa2f3378/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:12.288826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.288826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.289025 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.326869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] MFA Rules not processed for user `f87045df238146108914b050fa2f3378`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:12.328747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:12.368774 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:12.424248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1405/2808] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:12.439372 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:12.530708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b120-c636-4765-a0d8-155feea0f6cc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:12.536559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:12.539027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.539027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.539027 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.539351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:12.541528 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:12.661118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1404/2809] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:12 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:12.662947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:12.680011 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c2311-aa94-4420-bd32-eebc3a3aed09 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:12.795799 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:12.796976 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.799302 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.799764 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.800962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:12.804288 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:12.967949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1406/2810] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:12 2022] POST /identity/v3/users => generated 294 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:12.968936 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:12.983271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e588ec4-7fc1-46f9-ac43-a88f21c46734 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:12.989402 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:12.990229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:12.991066 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:12.991637 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:12.992362 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:12.994123 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:13.012566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:13.019389 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1405/2811] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:13.024716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f97b00-d81b-4df9-8eb2-f9082c788fe5 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:13.030579 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:13.031415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:13.036425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:13.036425 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] PATH_INFO: `/v3/projects/baed0533dbb44c89b4c91ca7c82341ca/users/7b33e727c7fa430888d5c17e40c67f44/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:13.047869 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baed0533dbb44c89b4c91ca7c82341ca, user_id=7b33e727c7fa430888d5c17e40c67f44, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:13.050810 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:13.132445 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1407/2812] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:13 2022] PUT /identity/v3/projects/baed0533dbb44c89b4c91ca7c82341ca/users/7b33e727c7fa430888d5c17e40c67f44/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:13.136480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:13.147440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6cf82-55bb-4d99-86a7-4a871f5f7e92 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:13.155089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:13.229406 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5493126-774e-4131-8d92-487a17b4213a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:13.230149 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5493126-774e-4131-8d92-487a17b4213a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:13.230703 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5493126-774e-4131-8d92-487a17b4213a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:13.231167 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5493126-774e-4131-8d92-487a17b4213a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:13.232180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5493126-774e-4131-8d92-487a17b4213a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:13.237314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5493126-774e-4131-8d92-487a17b4213a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:13.251919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1406/2813] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:13.256886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:13.271010 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27b1a-7298-4a7d-99cd-b8c6c47e6ac9 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:13.277410 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:13.278286 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:13.278782 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:13.279262 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] PATH_INFO: `/v3/projects/baed0533dbb44c89b4c91ca7c82341ca/users/7b33e727c7fa430888d5c17e40c67f44/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:13.285440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baed0533dbb44c89b4c91ca7c82341ca, user_id=7b33e727c7fa430888d5c17e40c67f44, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:13.289440 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:13.303589 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1408/2814] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:13 2022] PUT /identity/v3/projects/baed0533dbb44c89b4c91ca7c82341ca/users/7b33e727c7fa430888d5c17e40c67f44/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:13.314808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb7bc-7bdc-4588-8fd9-0d8ff11a3497 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:13.315448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb7bc-7bdc-4588-8fd9-0d8ff11a3497 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:13.316093 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb7bc-7bdc-4588-8fd9-0d8ff11a3497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:13.363851 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-127cb7bc-7bdc-4588-8fd9-0d8ff11a3497 None None] MFA Rules not processed for user `7b33e727c7fa430888d5c17e40c67f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:13.368976 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-127cb7bc-7bdc-4588-8fd9-0d8ff11a3497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:13.497526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1407/2815] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:13.526583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:13.529052 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:13.637654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696337-7623-4230-be2f-b30fe2469577 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:13.641680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:13.642424 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:13.642809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:13.643186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:13.726556 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:13.771252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:13.771252 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:13.785517 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1409/2816] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:34:13.859291 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e6886-a124-4790-bc64-35fb699520b1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:13.859651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e6886-a124-4790-bc64-35fb699520b1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:13.859958 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e6886-a124-4790-bc64-35fb699520b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:13.890221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9b6e6886-a124-4790-bc64-35fb699520b1 None None] MFA Rules not processed for user `cae338755bbb49b8911bfe2ef9d0ce09`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:13.891418 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9b6e6886-a124-4790-bc64-35fb699520b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:13.993440 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1408/2817] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:14.009642 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:14.011277 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:14.114923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51e99-b0d8-42e5-b380-7cdf122cb20c service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:14.124136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:14.127493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:14.127990 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:14.128687 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:14.131153 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:14.198939 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:14.200549 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:14.215055 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1410/2818] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:17.079382 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:17.079877 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:17.079988 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:17.112759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] MFA Rules not processed for user `e4d96e16d368455aa92e1cd62219f106`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:17.114212 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:17.124052 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1409/2819] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:20.455080 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:20.468270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b467c-ce8e-45df-9f0c-ca7577b7be1e service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:20.471946 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:20.472547 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:20.472826 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:20.473128 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:20.483466 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:20.484886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:20.494512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1411/2820] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:27.095011 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:30.490180 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=75 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:38.711001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:38.713183 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:38.774713 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82c540-9651-44e8-8e45-4eab51389694 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:38.778156 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:38.778769 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:38.779053 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:38.779343 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:38.781113 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:38.845842 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:38.847297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915b3d7-1ce8-447d-8df7-0a61aa8dcb6b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:38.857729 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1410/2821] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:47.792021 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:47.854792 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fc51b-b712-405e-a0e0-ff980eae7df3 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:47.859355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:47.859974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:47.860254 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:47.860525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] PATH_INFO: `/v3/users/f87045df238146108914b050fa2f3378` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:47.861653 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87045df238146108914b050fa2f3378)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:47.863597 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:48.173212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87045df238146108914b050fa2f3378'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:48.267306 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87045df238146108914b050fa2f3378'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:48.275271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87045df238146108914b050fa2f3378'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:48.275910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87045df238146108914b050fa2f3378'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:48.288110 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87045df238146108914b050fa2f3378'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:48.294533 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1412/2822] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:47 2022] DELETE /identity/v3/users/f87045df238146108914b050fa2f3378 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:48.299101 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:48.299599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:48.299715 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:48.319208 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:48.320431 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:48.377818 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1411/2823] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:48.388193 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:48.389699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:48.449540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7bb10-b7f5-4c00-b792-60a70e11abe9 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:48.453296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:48.453902 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:48.454184 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:48.454451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:48.464289 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:48.465744 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:48.475067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1413/2824] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:48.936451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:48.937889 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:48.952189 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf800-82c0-46d1-acae-aa9080cc3c50 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:48.958659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:48.958659 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:48.961314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:48.961314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] PATH_INFO: `/v3/projects/07d13793ff6d4f14b436bbced1aa4de2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:48.961314 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d13793ff6d4f14b436bbced1aa4de2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:48.962254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:49.098224 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:49.101271 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d13793ff6d4f14b436bbced1aa4de2', 'request_id': 'req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.107104 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d13793ff6d4f14b436bbced1aa4de2', 'request_id': 'req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.113526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d13793ff6d4f14b436bbced1aa4de2', 'request_id': 'req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.226192 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] The token cache is being invalidate because project 07d13793ff6d4f14b436bbced1aa4de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:49.227007 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.227853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.229371 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1412/2825] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:48 2022] DELETE /identity/v3/projects/07d13793ff6d4f14b436bbced1aa4de2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:49.233379 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:49.244736 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022797cb-ae3b-49fd-8ba5-4c660fa65af7 service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:49.262462 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:49.299474 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:49.300103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:49.306680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:49.306680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] PATH_INFO: `/v3/users/7b33e727c7fa430888d5c17e40c67f44` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:49.306680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b33e727c7fa430888d5c17e40c67f44)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:49.306680 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:49.520583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b33e727c7fa430888d5c17e40c67f44'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.592552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b33e727c7fa430888d5c17e40c67f44'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.599740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b33e727c7fa430888d5c17e40c67f44'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.600381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b33e727c7fa430888d5c17e40c67f44'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.613407 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6c67c9ea-abac-4431-9937-5a1203e851e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b33e727c7fa430888d5c17e40c67f44'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:49.619757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1414/2826] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:49 2022] DELETE /identity/v3/users/7b33e727c7fa430888d5c17e40c67f44 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:50.068505 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.070039 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:50.084451 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0f6c-82a0-40a9-b2f6-b16c362efee0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.142035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.142711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.142992 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.143269 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] PATH_INFO: `/v3/projects/baed0533dbb44c89b4c91ca7c82341ca` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.144381 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=baed0533dbb44c89b4c91ca7c82341ca)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:50.146323 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:50.255930 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baed0533dbb44c89b4c91ca7c82341ca', 'request_id': 'req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:50.262781 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baed0533dbb44c89b4c91ca7c82341ca', 'request_id': 'req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:50.269215 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baed0533dbb44c89b4c91ca7c82341ca', 'request_id': 'req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:50.387328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] The token cache is being invalidate because project baed0533dbb44c89b4c91ca7c82341ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:50.393285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:50.393285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93fe26bf-fd5a-4370-a7a5-e4115c9a1fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:50.393285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1413/2827] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:50 2022] DELETE /identity/v3/projects/baed0533dbb44c89b4c91ca7c82341ca => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:50.394651 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.395112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.395210 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.415122 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:50.416325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:50.474689 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1415/2828] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:50.483619 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.483909 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.484187 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.503696 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:50.504903 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:50.513575 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1414/2829] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:50.517712 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.527324 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b2cb3-835d-47b7-a5af-06bea8e08016 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.531809 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.532396 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.532670 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.532936 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.533615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:34:50.534190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:50.535751 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:50.547082 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1416/2830] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:34:50.551253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.561013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716440de-279e-44bb-a443-bde02c1971d7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.565551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.566141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.566419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.566691 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.567319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:50.569223 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:50.692385 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1415/2831] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:50 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:50.696665 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.706525 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc61dbc-0ced-458c-b1ee-5675d0267e17 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.732956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:50.763270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.763875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.764155 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.764428 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.765093 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:50.767022 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:50.877553 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1417/2832] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:50 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:50.881499 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.892626 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3b307-9d75-4693-a6e1-5d9195c0e86b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.897970 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.899789 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.900121 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.900363 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.901076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:50.905395 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:50.914282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1416/2833] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:50.918111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.928105 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aa13b-1ce3-45db-8442-28a1f0edd798 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.934243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c2a9a-6327-4728-aa0c-a37e6c237639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.934243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c2a9a-6327-4728-aa0c-a37e6c237639 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.934243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c2a9a-6327-4728-aa0c-a37e6c237639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.934243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c2a9a-6327-4728-aa0c-a37e6c237639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:50.934951 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c2a9a-6327-4728-aa0c-a37e6c237639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:50.936540 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c2a9a-6327-4728-aa0c-a37e6c237639 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:50.949654 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1418/2834] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:50.962846 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:50.974170 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4670-38f4-4ca7-9274-e77bff016dee admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:50.979583 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:50.980712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:50.980712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:50.980962 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] PATH_INFO: `/v3/projects/9f43d745fb6b471da8d1443f8bb87801/users/79eb9936e9ff407aacd069559839ad5d/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.001649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f43d745fb6b471da8d1443f8bb87801, user_id=79eb9936e9ff407aacd069559839ad5d, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.004419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.068600 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:51.074152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1417/2835] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:50 2022] PUT /identity/v3/projects/9f43d745fb6b471da8d1443f8bb87801/users/79eb9936e9ff407aacd069559839ad5d/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:51.084494 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.084782 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.085098 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.112331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] MFA Rules not processed for user `79eb9936e9ff407aacd069559839ad5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:51.113614 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:51.178484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1419/2836] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:51.183757 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.244131 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3230540-4fb9-4d0f-8b11-e14b980f06c3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.248807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.256313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.256313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.256313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.256313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.256313 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.337594 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1418/2837] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:51 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:34:51.341709 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.352969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3d049-4b63-4145-a47e-7ac4a585c22d None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.407949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.408606 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.408808 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.409362 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.409974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.411868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.516716 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1420/2838] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:51 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:51.520794 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.531654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a7bc5-9ecd-4d43-8bad-05585a28de03 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.536565 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.537231 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.537508 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.537775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.538415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.540103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.552535 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1419/2839] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:51.555161 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.565256 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883345f-cdf5-4756-a98c-e8394091df24 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.570616 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.571296 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.571584 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.571879 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] PATH_INFO: `/v3/projects/8ba59d8d279043398b98697f5b488e5f/users/4fc908b9da3847c0b5620d9f5a716372/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.588179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba59d8d279043398b98697f5b488e5f, user_id=4fc908b9da3847c0b5620d9f5a716372, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.590877 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.694414 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1421/2840] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:51 2022] PUT /identity/v3/projects/8ba59d8d279043398b98697f5b488e5f/users/4fc908b9da3847c0b5620d9f5a716372/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:51.699055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.709141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42bff-36fe-4cb7-a367-0424a0750e7b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.765620 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb7acf-cddf-458c-95ec-c05654caef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.766219 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb7acf-cddf-458c-95ec-c05654caef81 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.766498 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb7acf-cddf-458c-95ec-c05654caef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.766820 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb7acf-cddf-458c-95ec-c05654caef81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.767396 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb7acf-cddf-458c-95ec-c05654caef81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.768978 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb7acf-cddf-458c-95ec-c05654caef81 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.779068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1420/2841] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:51.782952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.788851 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:51.794342 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3fefe-bd28-436a-868f-0c427e30f2ab admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.798986 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.799657 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.799937 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.800821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] PATH_INFO: `/v3/projects/8ba59d8d279043398b98697f5b488e5f/users/4fc908b9da3847c0b5620d9f5a716372/roles/dd76871a868143b482c4982027fdc1f5` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.802186 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba59d8d279043398b98697f5b488e5f, user_id=4fc908b9da3847c0b5620d9f5a716372, role_id=dd76871a868143b482c4982027fdc1f5)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:51.805361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:51.815381 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1422/2842] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:51 2022] PUT /identity/v3/projects/8ba59d8d279043398b98697f5b488e5f/users/4fc908b9da3847c0b5620d9f5a716372/roles/dd76871a868143b482c4982027fdc1f5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:51.826520 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.826812 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.827117 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:51.856076 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] MFA Rules not processed for user `4fc908b9da3847c0b5620d9f5a716372`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:51.857316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:51.916380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:51.924935 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1421/2843] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:51.943945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:51.943945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:51.994702 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02237f-32fe-410b-bf9e-aed95fc61495 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:51.998570 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d6964-814a-466f-9795-059b375bcd13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:51.999170 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d6964-814a-466f-9795-059b375bcd13 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:51.999450 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d6964-814a-466f-9795-059b375bcd13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:51.999721 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d6964-814a-466f-9795-059b375bcd13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:52.008909 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d6964-814a-466f-9795-059b375bcd13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:52.010361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d6964-814a-466f-9795-059b375bcd13 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:52.020111 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1423/2844] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:52.050013 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:52.113304 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91f584-d412-49e9-9017-ebde7816163c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:52.118248 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:52.118887 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:52.119229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:52.119439 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] PATH_INFO: `/v3/users/79eb9936e9ff407aacd069559839ad5d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:52.120727 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79eb9936e9ff407aacd069559839ad5d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:52.122671 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:52.301906 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79eb9936e9ff407aacd069559839ad5d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:52.309564 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79eb9936e9ff407aacd069559839ad5d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:52.369678 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79eb9936e9ff407aacd069559839ad5d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:52.370308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79eb9936e9ff407aacd069559839ad5d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:52.384872 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79eb9936e9ff407aacd069559839ad5d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:52.390915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1422/2845] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:52 2022] DELETE /identity/v3/users/79eb9936e9ff407aacd069559839ad5d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:52.395452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:52.395742 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:52.396076 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:52.416089 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:52.417331 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:52.474690 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1424/2846] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:52.485130 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:52.486514 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:52.544893 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d475ab-4c52-4e94-ba5d-8a778d16951a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:52.548590 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:52.549220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:52.549500 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:52.549773 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:52.558707 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:52.560069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:52.569285 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1423/2847] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:53.011501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:53.011501 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:53.026922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b8125-06e1-4a15-aa06-dbd04edcec35 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:53.034955 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:53.035604 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:53.035886 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:53.036263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] PATH_INFO: `/v3/projects/9f43d745fb6b471da8d1443f8bb87801` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:53.037666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f43d745fb6b471da8d1443f8bb87801)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:53.040071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:53.137563 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f43d745fb6b471da8d1443f8bb87801', 'request_id': 'req-3b0ab585-084d-4bb9-b774-8d16c6a84072'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.144493 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f43d745fb6b471da8d1443f8bb87801', 'request_id': 'req-3b0ab585-084d-4bb9-b774-8d16c6a84072'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.152403 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f43d745fb6b471da8d1443f8bb87801', 'request_id': 'req-3b0ab585-084d-4bb9-b774-8d16c6a84072'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.226714 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] The token cache is being invalidate because project 9f43d745fb6b471da8d1443f8bb87801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:53.227112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.227819 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.229416 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1425/2848] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:53 2022] DELETE /identity/v3/projects/9f43d745fb6b471da8d1443f8bb87801 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:53.233596 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:53.244683 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cf9e5-3687-42f5-bbaa-3307380fc808 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:53.258599 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:53.306202 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:53.306844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:53.307128 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:53.307404 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] PATH_INFO: `/v3/users/4fc908b9da3847c0b5620d9f5a716372` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:53.308543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc908b9da3847c0b5620d9f5a716372)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:53.310482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:53.419330 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc908b9da3847c0b5620d9f5a716372'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.426448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc908b9da3847c0b5620d9f5a716372'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.507037 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc908b9da3847c0b5620d9f5a716372'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.508357 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc908b9da3847c0b5620d9f5a716372'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.522095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-19c23373-dd27-4715-8e2b-0eff0dc59804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc908b9da3847c0b5620d9f5a716372'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:53.528779 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1424/2849] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:53 2022] DELETE /identity/v3/users/4fc908b9da3847c0b5620d9f5a716372 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:53.932035 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:53.933538 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:53.947521 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab585-084d-4bb9-b774-8d16c6a84072 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:53.960360 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:54.006336 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.006956 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.007230 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.007511 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] PATH_INFO: `/v3/projects/8ba59d8d279043398b98697f5b488e5f` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.009406 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba59d8d279043398b98697f5b488e5f)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.011279 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.170215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba59d8d279043398b98697f5b488e5f', 'request_id': 'req-507e8325-435d-42e8-bb6a-241b348a06b7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:54.175615 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba59d8d279043398b98697f5b488e5f', 'request_id': 'req-507e8325-435d-42e8-bb6a-241b348a06b7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:54.181669 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba59d8d279043398b98697f5b488e5f', 'request_id': 'req-507e8325-435d-42e8-bb6a-241b348a06b7'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:54.265922 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] The token cache is being invalidate because project 8ba59d8d279043398b98697f5b488e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80227) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:54.266212 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:54.266832 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-507e8325-435d-42e8-bb6a-241b348a06b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:54.268308 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1426/2850] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:53 2022] DELETE /identity/v3/projects/8ba59d8d279043398b98697f5b488e5f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:54.275242 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.275760 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.275867 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.295744 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:54.296934 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:54.353733 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1425/2851] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:54.362345 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.362637 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.362916 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.383974 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:54.385199 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:54.393103 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1427/2852] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:54.397380 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:54.407679 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a350c37-bcc8-4f81-bb5a-14fd7d2dbb1c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:54.412577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.413211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.413493 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.413807 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.414459 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:34:54.414764 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.416341 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.433913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1426/2853] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:34:54.435027 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:54.444320 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50bee-61da-4684-b0f5-2aeacaba2a91 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:54.450598 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.451354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.451629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.451895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] PATH_INFO: `/v3/projects` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.452518 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.454467 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.525060 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1428/2854] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:54 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:54.529072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:54.539915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005a3da-e32c-4e27-b79c-341703d0e95f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:54.567058 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:54.603853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.604492 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.604775 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.605072 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.605709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.607608 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.696543 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1427/2855] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:54 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:54.700409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:54.711559 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2beb5-bf4e-4e62-9658-bb527ef65cff admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:54.716326 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.716969 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.717298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.717573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.718213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.719805 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.734056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1429/2856] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:34:54.735318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:54.747336 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ccbc6-4912-4aa8-a9b1-7310c5843831 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:54.752317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc068c7-7da6-43c1-9e7e-9ccd41b4f339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.752932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc068c7-7da6-43c1-9e7e-9ccd41b4f339 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.753247 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc068c7-7da6-43c1-9e7e-9ccd41b4f339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.753523 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc068c7-7da6-43c1-9e7e-9ccd41b4f339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.754169 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc068c7-7da6-43c1-9e7e-9ccd41b4f339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.755772 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc068c7-7da6-43c1-9e7e-9ccd41b4f339 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.765879 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1428/2857] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:54.770078 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:54.781020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10759d45-6c15-4a64-a58e-c293270f3866 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:54.785931 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:54.786573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] REQUEST_METHOD: `PUT` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.786857 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.787135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] PATH_INFO: `/v3/projects/74519deb5ef24d7298c86db2adda7116/users/6aa595ba6e1a49f48dfb2509cc4c998e/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.802109 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] RBAC: Authorizing `identity:create_grant(project_id=74519deb5ef24d7298c86db2adda7116, user_id=6aa595ba6e1a49f48dfb2509cc4c998e, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:54.804807 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:54.901476 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1430/2858] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:54 2022] PUT /identity/v3/projects/74519deb5ef24d7298c86db2adda7116/users/6aa595ba6e1a49f48dfb2509cc4c998e/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:54.911714 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:54.911998 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:54.912272 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:54.942210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] MFA Rules not processed for user `6aa595ba6e1a49f48dfb2509cc4c998e`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:54.943423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:55.011954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1429/2859] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:55.023699 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:55.025624 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:55.076906 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:55.089661 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd1792-9bd8-482e-9e9f-0de3bc83535b admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:55.093458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:55.094070 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:55.094355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:55.094629 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:55.103947 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:55.105541 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:55.114520 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1431/2860] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:57.078410 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:57.079913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:57.142018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff4d48-3153-4ad1-827f-18823f9beb64 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:57.145768 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:57.146370 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:57.146660 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:57.146932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:57.150468 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:57.197068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:57.214690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:57.216112 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084af86d-0001-4965-884a-24a86e11a98c service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:57.225582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1430/2861] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:58.213626 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:58.275848 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b537ae-22a5-43ec-a7ad-53c8b2676c23 service cinder] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:58.280404 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:58.281054 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:58.281340 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:58.281613 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] PATH_INFO: `/v3/users/6aa595ba6e1a49f48dfb2509cc4c998e` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:58.282731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa595ba6e1a49f48dfb2509cc4c998e)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:58.284920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:58.464872 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa595ba6e1a49f48dfb2509cc4c998e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:58.535758 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa595ba6e1a49f48dfb2509cc4c998e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:58.543740 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa595ba6e1a49f48dfb2509cc4c998e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:58.545215 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa595ba6e1a49f48dfb2509cc4c998e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:58.556756 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa595ba6e1a49f48dfb2509cc4c998e'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:58.562577 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1432/2862] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:58 2022] DELETE /identity/v3/users/6aa595ba6e1a49f48dfb2509cc4c998e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:58.567080 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:58.567876 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:58.568148 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:58.588571 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:58.589849 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:58.653798 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1431/2863] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:58.664504 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:58.666770 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:58.728452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3f10-6cd5-41d8-a93a-bac75de7cb15 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:58.731988 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d1b2-dafc-430d-b3c8-724909eabdd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:58.732592 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d1b2-dafc-430d-b3c8-724909eabdd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:58.732867 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d1b2-dafc-430d-b3c8-724909eabdd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:58.733329 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d1b2-dafc-430d-b3c8-724909eabdd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:58.743885 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d1b2-dafc-430d-b3c8-724909eabdd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:58.745524 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d1b2-dafc-430d-b3c8-724909eabdd2 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:58.755354 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1433/2864] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:59.205018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:59.205018 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:59.224799 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e475b53-6c44-4677-ab1b-c362a557901d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:59.233067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:59.233067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.233067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.233067 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] PATH_INFO: `/v3/projects/74519deb5ef24d7298c86db2adda7116` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.234883 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74519deb5ef24d7298c86db2adda7116)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:59.237095 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:59.337617 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74519deb5ef24d7298c86db2adda7116', 'request_id': 'req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:59.344262 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74519deb5ef24d7298c86db2adda7116', 'request_id': 'req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:59.352686 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74519deb5ef24d7298c86db2adda7116', 'request_id': 'req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:59.428211 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] The token cache is being invalidate because project 74519deb5ef24d7298c86db2adda7116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:59.428518 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:59.429319 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-54627f84-d6ae-4401-bb5a-1b3a6267a6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:59.431094 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1432/2865] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:59 2022] DELETE /identity/v3/projects/74519deb5ef24d7298c86db2adda7116 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:59.435847 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.436358 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.436490 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.450762 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:59.457016 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:59.458259 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:59.520945 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1434/2866] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:59.529491 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.534700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.534700 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.551759 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:59.553057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:59.562308 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1433/2867] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:59.566648 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:59.577647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d80d-91f0-41c9-96cb-020f6ce9dbaf None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:59.582325 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:59.582932 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.583213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.583486 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] PATH_INFO: `/v3/domains` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.584137 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80227) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:34:59.584442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:59.593893 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:59.610053 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1435/2868] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 10 13:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:34:59.611280 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:59.621306 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1288e59-45fb-4735-9ec2-041b98bb6970 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:59.626253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:59.626875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.627152 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.627423 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.628069 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:59.630028 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:59.737392 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1434/2869] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:59 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:59.738526 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:59.751456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3147b-ab7c-4f55-bebc-36437336254c admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:59.823373 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:59.824087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.824442 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.824788 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] PATH_INFO: `/v3/users` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.825583 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:59.827585 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:59.908417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1436/2870] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:59 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:59.912834 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:59.928737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017e8c6-1e9e-4e13-bf52-1aafb0ac9ee4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:59.931302 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:59.931933 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.932214 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.932482 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.933253 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:59.934895 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:59.948305 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1435/2871] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:34:59.952248 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:59.964051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bee7e6-0603-475d-b469-d3c8732c83ca admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:59.970349 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62e883-ce6e-4795-a25d-7a5d4c3fbf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:59.971112 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62e883-ce6e-4795-a25d-7a5d4c3fbf51 admin admin] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:59.971495 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62e883-ce6e-4795-a25d-7a5d4c3fbf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:59.972129 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62e883-ce6e-4795-a25d-7a5d4c3fbf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:59.972895 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62e883-ce6e-4795-a25d-7a5d4c3fbf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:59.974952 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62e883-ce6e-4795-a25d-7a5d4c3fbf51 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:59.989135 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1437/2872] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:59.994092 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:00.004526 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd852a-de52-4c36-ba18-941bbf7b0e56 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:00.009718 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:00.010401 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:00.010774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:00.011055 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] PATH_INFO: `/v3/projects/eebf57e457864319ba8d6adf6e24debf/users/526b90e5b2054247b65078f7875b5af1/roles/31c6a032fe034fd5abc43cc4ce065209` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:00.029318 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebf57e457864319ba8d6adf6e24debf, user_id=526b90e5b2054247b65078f7875b5af1, role_id=31c6a032fe034fd5abc43cc4ce065209)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:00.032332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:00.154959 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1436/2873] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:59 2022] PUT /identity/v3/projects/eebf57e457864319ba8d6adf6e24debf/users/526b90e5b2054247b65078f7875b5af1/roles/31c6a032fe034fd5abc43cc4ce065209 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:35:00.220860 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:00.221174 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:00.221451 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:00.249768 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] MFA Rules not processed for user `526b90e5b2054247b65078f7875b5af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:00.250998 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:00.320552 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1438/2874] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:35:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:00.331875 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:00.333651 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:00.339680 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:00.391229 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b91d2f-837b-410d-88e0-440ba83fc040 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:00.394892 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9d46-921a-4563-a789-75d42b4cd4b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8279c74cfda94068b5b392123282e556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:00.395497 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9d46-921a-4563-a789-75d42b4cd4b5 service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:00.395777 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9d46-921a-4563-a789-75d42b4cd4b5 service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:00.396048 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9d46-921a-4563-a789-75d42b4cd4b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:00.405035 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac9d46-921a-4563-a789-75d42b4cd4b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:00.406448 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac9d46-921a-4563-a789-75d42b4cd4b5 service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:00.415701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1437/2875] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:10.317357 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:10.438435 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=25 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:28.646298 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:28.709891 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a6b2-b94d-4925-b678-13410555ffa8 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:28.714586 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:28.715214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:28.715508 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:28.716147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] PATH_INFO: `/v3/users/526b90e5b2054247b65078f7875b5af1` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:28.716966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=526b90e5b2054247b65078f7875b5af1)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:28.718910 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:28.942830 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526b90e5b2054247b65078f7875b5af1'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.020452 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526b90e5b2054247b65078f7875b5af1'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.027765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526b90e5b2054247b65078f7875b5af1'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.028388 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526b90e5b2054247b65078f7875b5af1'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.041051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526b90e5b2054247b65078f7875b5af1'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.047002 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1439/2876] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:35:28 2022] DELETE /identity/v3/users/526b90e5b2054247b65078f7875b5af1 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:35:29.051480 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:29.051996 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:29.057873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:29.071763 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] MFA Rules not processed for user `40886c602f2b4c218564e5847aed724b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:29.073124 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:29.132455 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1438/2877] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:29.142271 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:29.143667 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:29.201049 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b737941-eb83-4b21-ad28-b9fc0fb3bd15 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:29.204263 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:29.204868 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:29.205190 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:29.205468 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:29.215142 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:29.216566 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:29.225975 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1440/2878] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:35:29.657673 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:29.659701 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:29.672921 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3550c7a-f19d-4687-9713-f5036cafeae2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:29.677624 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:29.678267 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:29.678549 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:29.678827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] PATH_INFO: `/v3/projects/eebf57e457864319ba8d6adf6e24debf` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:29.679960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebf57e457864319ba8d6adf6e24debf)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:29.682256 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:29.797804 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebf57e457864319ba8d6adf6e24debf', 'request_id': 'req-1b8889ff-083f-47c2-ada9-c1827c1af3ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.803151 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebf57e457864319ba8d6adf6e24debf', 'request_id': 'req-1b8889ff-083f-47c2-ada9-c1827c1af3ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.809603 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebf57e457864319ba8d6adf6e24debf', 'request_id': 'req-1b8889ff-083f-47c2-ada9-c1827c1af3ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.882692 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] The token cache is being invalidate because project eebf57e457864319ba8d6adf6e24debf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:35:29.882961 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.883604 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:35:29.885611 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1439/2879] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:35:29 2022] DELETE /identity/v3/projects/eebf57e457864319ba8d6adf6e24debf => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:35:39.221831 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:39.879645 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:36:30.275355 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:30.284409 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b850-136e-4ff1-adc0-e89979bd49bc service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:30.333294 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:30.333890 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:30.334166 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:30.334432 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:30.336130 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:30.345971 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:30.347456 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:30.356056 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1441/2880] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:36:40.351923 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=26 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:36:44.554761 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:44.556331 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:44.568582 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8889ff-083f-47c2-ada9-c1827c1af3ec admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:44.572198 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:44.573623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:44.573623 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:44.573915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:44.575709 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:44.647504 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:44.648947 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:44.661551 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1440/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:36:45.840214 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:45.842875 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:45.852573 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6237a53-31f0-463b-b178-08a26e6a019c service neutron] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:45.857051 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:45.857682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:45.857966 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:45.858238 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:45.859933 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:45.873449 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:45.874856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffb427-dbe5-44b3-a8c7-c62b7425b0d2 service neutron] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:45.886104 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1442/2882] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:36:47.280476 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:47.283150 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:47.298368 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad3c67-d647-4ba4-839e-0fcb02ad0a01 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:47.304328 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:47.304954 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:47.305277 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:47.305708 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:47.307957 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:47.327982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:47.327982 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:47.339986 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1441/2883] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:36:49.294006 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:49.294480 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:49.294647 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:49.314073 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] MFA Rules not processed for user `9675d83e26b647489e4b6195bb814aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:36:49.315412 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:49.371141 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1443/2884] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:36:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:36:49.693559 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:49.703161 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a7907-6f7d-43a7-95b0-3108eefaecf6 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:49.706711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:49.707300 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:49.707576 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:49.707844 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:49.709569 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:49.720176 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:49.721753 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:49.730446 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1442/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:36:51.465958 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:51.467337 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:51.536658 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629375b0-7d6d-48c9-b15a-f7374cb5de95 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:51.542484 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:51.543087 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:51.543478 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:51.543757 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:51.545682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:51.558274 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:51.559856 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:51.570004 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1444/2886] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:36:59.726180 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=46 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:01.565691 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=63 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:09.522915 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:09.528808 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:09.541106 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca409-9df7-41da-99b6-807322a45e7c service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:09.546577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:09.550103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:09.550103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:09.550103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:09.550103 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:09.562394 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:09.564006 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:09.575001 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1443/2887] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:19.569902 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:37.602858 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:37.614211 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9653e-f7b3-4bbb-94a0-b91588a94068 service placement] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:37.619257 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:37.619920 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:37.620204 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:37.620483 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:37.622223 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:37.634020 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:37.637855 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d55457-141d-4cdf-ab62-1a64a15519c0 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:37.645602 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1445/2888] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:47.641361 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=6 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:54.354960 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.356367 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:54.412478 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc7bd-7119-4510-b63e-54a8f9938b9d service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.416057 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ff5fa2e326485bb2f915fa4f8870fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.416654 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.416932 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.417234 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.418913 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:54.429433 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.430873 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.439534 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1444/2889] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:03.000196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] REQUEST_METHOD: `POST` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.000745 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.000914 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.020168 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.auth.core [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] MFA Rules not processed for user `9675d83e26b647489e4b6195bb814aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80227) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:03.021415 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.028979 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1446/2890] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 10 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:04.435496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=26 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:04.857774 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.868316 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723f3e2-074a-40ff-8b8f-f4acae053298 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.873813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.873813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.873813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.873813 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.875200 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.937990 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.939419 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.948928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1445/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:09.948181 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:09.961232 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831b129-c1e6-4831-a3da-de6dd64b66f1 None None] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:09.966253 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:09.966859 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:09.967138 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:09.967411 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:09.970376 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:09.982391 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:09.984731 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:09.995196 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1447/2892] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:13.827924 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:13.839157 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43034c88-2264-48a6-be38-a268bfeda1b0 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:13.844731 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:13.845366 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.845657 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.845928 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.847684 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:13.859424 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:13.861220 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:13.871494 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1446/2893] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:13.946686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:13.948136 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:13.962625 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb18dd-4788-4ff4-acc3-39a12b637595 service placement] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:13.967555 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:13.968926 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.969251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.969527 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] PATH_INFO: `/v3/users/e4d96e16d368455aa92e1cd62219f106` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.970686 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d96e16d368455aa92e1cd62219f106)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:13.972610 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.153765 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d96e16d368455aa92e1cd62219f106'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.228747 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d96e16d368455aa92e1cd62219f106'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.237156 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d96e16d368455aa92e1cd62219f106'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.237793 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d96e16d368455aa92e1cd62219f106'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.251023 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d96e16d368455aa92e1cd62219f106'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.256870 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1448/2894] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:13 2022] DELETE /identity/v3/users/e4d96e16d368455aa92e1cd62219f106 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:14.493239 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.504690 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cc8f7-e08d-4509-9c10-9cbd669669cf service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.561995 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.562634 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.562917 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.563190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] PATH_INFO: `/v3/projects/1596bc6318b24822beda3dc89834c7b2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.564332 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1596bc6318b24822beda3dc89834c7b2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.566295 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.683087 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1596bc6318b24822beda3dc89834c7b2', 'request_id': 'req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.690351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1596bc6318b24822beda3dc89834c7b2', 'request_id': 'req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.697711 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1596bc6318b24822beda3dc89834c7b2', 'request_id': 'req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.798800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] The token cache is being invalidate because project 1596bc6318b24822beda3dc89834c7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:14.799232 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.800257 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:14.802400 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1447/2895] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:14 2022] DELETE /identity/v3/projects/1596bc6318b24822beda3dc89834c7b2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:14.806417 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.809139 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:14.825251 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2637-69b5-4ad1-9ce3-a710b4dcceb3 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.895785 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.896530 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.896884 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.897280 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] PATH_INFO: `/v3/users/cae338755bbb49b8911bfe2ef9d0ce09` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.898496 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae338755bbb49b8911bfe2ef9d0ce09)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.900565 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.129512 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae338755bbb49b8911bfe2ef9d0ce09'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.203594 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae338755bbb49b8911bfe2ef9d0ce09'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.204938 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:15.211147 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae338755bbb49b8911bfe2ef9d0ce09'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.212235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae338755bbb49b8911bfe2ef9d0ce09'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.224179 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae338755bbb49b8911bfe2ef9d0ce09'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.229982 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1449/2896] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:14 2022] DELETE /identity/v3/users/cae338755bbb49b8911bfe2ef9d0ce09 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:15.467969 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:15.469408 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:15.484254 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed5c4-3daf-42aa-ad70-d08ba46f5cdd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:15.492783 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:15.542415 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:15.543049 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:15.543353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:15.543649 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] PATH_INFO: `/v3/projects/5817f311a0c0410db02c9da5e334e67c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:15.544800 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5817f311a0c0410db02c9da5e334e67c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:15.547297 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.673827 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5817f311a0c0410db02c9da5e334e67c', 'request_id': 'req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.679221 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5817f311a0c0410db02c9da5e334e67c', 'request_id': 'req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.685351 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5817f311a0c0410db02c9da5e334e67c', 'request_id': 'req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.745190 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] The token cache is being invalidate because project 5817f311a0c0410db02c9da5e334e67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:15.745716 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.746432 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.747919 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1448/2897] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:15 2022] DELETE /identity/v3/projects/5817f311a0c0410db02c9da5e334e67c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:25.248535 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:25.251961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:25.329213 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468d19-99c3-4749-a42c-c941c0b75cc2 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:25.394284 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dda95d65ff54916b0565d412c78e75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:25.394791 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:25.395071 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:25.395339 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:25.397176 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:25.410682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:25.412265 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:25.423458 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1450/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:25.751089 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:35.418502 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=73,keystone:DELETE=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:21.039496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:21.040926 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:21.098141 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd380c-2eb1-4ba6-b2f0-fd909806fd8b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:21.101712 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4548fbe5b418ea9787d78e40838a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:21.102310 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:21.102585 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:21.102853 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:21.104557 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:21.168068 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:21.169443 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:21.179854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1449/2899] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:41:28.225961 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:28.227546 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:28.243705 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4acb90-f96e-45e8-967f-acb50065e03d service nova] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:28.248273 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:28.248949 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:28.249164 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:28.249436 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] PATH_INFO: `/v3/users/9675d83e26b647489e4b6195bb814aab` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:28.250561 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9675d83e26b647489e4b6195bb814aab)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:28.253798 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:28.426682 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9675d83e26b647489e4b6195bb814aab'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.498402 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9675d83e26b647489e4b6195bb814aab'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.506576 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9675d83e26b647489e4b6195bb814aab'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.507202 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9675d83e26b647489e4b6195bb814aab'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.520243 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9675d83e26b647489e4b6195bb814aab'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.526268 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1451/2900] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:41:28 2022] DELETE /identity/v3/users/9675d83e26b647489e4b6195bb814aab => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:41:28.786108 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:28.796201 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e66cb-7fdb-41f9-9cff-81646553700b service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:28.854282 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:28.854914 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:28.855194 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:28.855467 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] PATH_INFO: `/v3/projects/ded676e84f4145d5912a3878ef9af132` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:28.856577 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded676e84f4145d5912a3878ef9af132)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:28.858502 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:28.959296 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded676e84f4145d5912a3878ef9af132', 'request_id': 'req-26c550d7-81e4-4f97-b330-522e42dcaf35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.961210 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:28.965222 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded676e84f4145d5912a3878ef9af132', 'request_id': 'req-26c550d7-81e4-4f97-b330-522e42dcaf35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:28.971496 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded676e84f4145d5912a3878ef9af132', 'request_id': 'req-26c550d7-81e4-4f97-b330-522e42dcaf35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.057530 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] The token cache is being invalidate because project ded676e84f4145d5912a3878ef9af132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:41:29.057737 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.058353 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.059854 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1450/2901] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:41:28 2022] DELETE /identity/v3/projects/ded676e84f4145d5912a3878ef9af132 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:41:29.064666 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:29.066270 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:29.079067 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f2671-346e-4ce3-8a33-3ace73859704 admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:29.134548 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:29.135172 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:29.135453 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:29.135749 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] PATH_INFO: `/v3/users/8076a76979f048359da245ac01300429` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:29.136876 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8076a76979f048359da245ac01300429)` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:29.138821 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:29.304720 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8076a76979f048359da245ac01300429'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.372722 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8076a76979f048359da245ac01300429'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.380940 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8076a76979f048359da245ac01300429'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.383235 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8076a76979f048359da245ac01300429'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.389550 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:29.394619 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.notifications [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8076a76979f048359da245ac01300429'} {{(pid=80227) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.400655 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1452/2902] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 10 13:41:29 2022] DELETE /identity/v3/users/8076a76979f048359da245ac01300429 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:41:29.661740 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:29.663236 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:29.676517 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c550d7-81e4-4f97-b330-522e42dcaf35 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:29.737317 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40886c602f2b4c218564e5847aed724b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55c6cff78b754392afc788fabbf074a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:29.737952 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:29.738230 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:29.738503 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] PATH_INFO: `/v3/projects/39593d796c77478b990a9d5bc90e176d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:29.739607 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39593d796c77478b990a9d5bc90e176d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:29.741581 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:29.852816 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39593d796c77478b990a9d5bc90e176d', 'request_id': 'req-86211f83-9420-41a2-92ac-cb61c34f6114'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.858299 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39593d796c77478b990a9d5bc90e176d', 'request_id': 'req-86211f83-9420-41a2-92ac-cb61c34f6114'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.864483 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39593d796c77478b990a9d5bc90e176d', 'request_id': 'req-86211f83-9420-41a2-92ac-cb61c34f6114'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.938118 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] The token cache is being invalidate because project 39593d796c77478b990a9d5bc90e176d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:41:29.938376 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.940063 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-86211f83-9420-41a2-92ac-cb61c34f6114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:41:29.942335 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1451/2903] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 10 13:41:29 2022] DELETE /identity/v3/projects/39593d796c77478b990a9d5bc90e176d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:41:39.627539 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:39.934814 dv-cinder-836027-5-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:44.550608 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Authenticating user token {{(pid=80227) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:44.552064 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:44.691995 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823a90b-6c83-4bf5-bb36-4450f5c78c0d admin admin] Validating token access rules against request {{(pid=80227) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:44.751065 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c82e23142a343b096a493e01071a506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc243b143dab48d7a19359dc314c43c2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80227) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:44.751678 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] REQUEST_METHOD: `GET` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:44.751962 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] SCRIPT_NAME: `/identity` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:44.752237 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80227) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:44.754088 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.fernet_utils [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80227) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:44.765229 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:44.766800 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6d49f-78ed-45e9-836b-0149e2678824 service placement] RBAC: Authorization granted {{(pid=80227) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:44.776697 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: [pid: 80227|app: 0|req: 1453/2904] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:54.772500 dv-cinder-836027-5-iscsi devstack@keystone.service[80227]: DEBUG dbcounter [-] [80227] Writing DB stats keystone:SELECT=89 {{(pid=80227) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}